Script 'mail_helper' called by obssrc
Hello community,
here is the log from the commit of package sesdev for openSUSE:Factory checked
in at 2021-03-24 16:11:48
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Comparing /work/SRC/openSUSE:Factory/sesdev (Old)
and /work/SRC/openSUSE:Factory/.sesdev.new.2401 (New)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "sesdev"
Wed Mar 24 16:11:48 2021 rev:20 rq:880418 version:1.12.0+1616368506.g9bba186
Changes:
--------
--- /work/SRC/openSUSE:Factory/sesdev/sesdev.changes 2020-12-11
20:16:35.952656745 +0100
+++ /work/SRC/openSUSE:Factory/.sesdev.new.2401/sesdev.changes 2021-03-24
16:11:49.707868258 +0100
@@ -1,0 +2,21 @@
+Sun Mar 21 23:15:13 UTC 2021 - Nathan Cutler <ncut...@suse.com>
+
+- Update to 1.12.0+1616368506.g9bba186:
+ + upstream 1.12.0 release (2021-03-22)
+ * provision: enable coredumps (PR #566)
+ * sesdev: support custom repo URLs pointing to .repo file (PR #571)
+ * cephadm: add 'period update' in radosgw deployment (PR #579)
+ * tox: add support for Python 3.8 (PR #582)
+ * sesdev.spec: fix missing qa/common/rgw.sh (PR #572)
+ * contrib/upgrade-demo-ses6-to-ses7.sh: fix (PR #573)
+ * zypper: be more careful when installing systemd-coredump (PR #577)
+ * sesdev: replace fancy list comprehension with simpleton code (PR #581)
+ * deployment: saner processing of --deepsea-repo, --deepsea-branch (PR
#584)
+ * zypper.j2: remove Python 2 even on SLE-12-SP3 (PR #585)
+ * provision.sh: persist systemd journal in SES7 only (PR #569)
+ * ceph-salt: no cephadm for nodes with no sesdev roles (PR #570)
+ * constant.py: drop nfs from octopus default roles (PR #587)
+ * nautilus_pre_stage_4.sh.j2: adapt to syntax change in Nautilus 14.2.17
(PR #589)
+ * qa/common/json.sh: adapt to change in ceph status json format (PR #590)
+
+-------------------------------------------------------------------
Old:
----
sesdev-1.11.0+1607012263.g73dea97.tar.gz
New:
----
sesdev-1.12.0+1616368506.g9bba186.tar.gz
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Other differences:
------------------
++++++ sesdev.spec ++++++
--- /var/tmp/diff_new_pack.7VPI1O/_old 2021-03-24 16:11:50.271868849 +0100
+++ /var/tmp/diff_new_pack.7VPI1O/_new 2021-03-24 16:11:50.275868853 +0100
@@ -16,7 +16,7 @@
#
Name: sesdev
-Version: 1.11.0+1607012263.g73dea97
+Version: 1.12.0+1616368506.g9bba186
Release: 1%{?dist}
Summary: CLI tool to deploy and manage SES clusters
License: MIT
@@ -92,6 +92,7 @@
install -m 0644 qa/common/common.sh
%{buildroot}/%{_datadir}/%{name}/qa/common/common.sh
install -m 0644 qa/common/helper.sh
%{buildroot}/%{_datadir}/%{name}/qa/common/helper.sh
install -m 0644 qa/common/json.sh
%{buildroot}/%{_datadir}/%{name}/qa/common/json.sh
+install -m 0644 qa/common/rgw.sh
%{buildroot}/%{_datadir}/%{name}/qa/common/rgw.sh
install -m 0644 qa/common/zypper.sh
%{buildroot}/%{_datadir}/%{name}/qa/common/zypper.sh
%files
++++++ sesdev-1.11.0+1607012263.g73dea97.tar.gz ->
sesdev-1.12.0+1616368506.g9bba186.tar.gz ++++++
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/sesdev-1.11.0+1607012263.g73dea97/CHANGELOG.md
new/sesdev-1.12.0+1616368506.g9bba186/CHANGELOG.md
--- old/sesdev-1.11.0+1607012263.g73dea97/CHANGELOG.md 2020-12-03
17:17:43.807857359 +0100
+++ new/sesdev-1.12.0+1616368506.g9bba186/CHANGELOG.md 2021-03-22
00:15:06.135033744 +0100
@@ -7,6 +7,30 @@
## [Unreleased]
+## [1.12.0] - 2021-03-22
+
+### Added
+- provision: enable coredumps (PR #566)
+- sesdev: support custom repo URLs pointing to .repo file (PR #571)
+- cephadm: add 'period update' in radosgw deployment (PR #579)
+- tox: add support for Python 3.8 (PR #582)
+
+### Fixed
+- sesdev.spec: fix missing qa/common/rgw.sh (PR #572)
+- contrib/upgrade-demo-ses6-to-ses7.sh: fix (PR #573)
+- zypper: be more careful when installing systemd-coredump (PR #577)
+- sesdev: replace fancy list comprehension with simpleton code (PR #581)
+- deployment: saner processing of --deepsea-repo, --deepsea-branch (PR #584)
+- zypper.j2: remove Python 2 even on SLE-12-SP3 (PR #585)
+
+### Changed
+- provision.sh: persist systemd journal in SES7 only (PR #569)
+- ceph-salt: no cephadm for nodes with no sesdev roles (PR #570)
+- constant.py: drop nfs from octopus default roles (PR #587)
+- nautilus_pre_stage_4.sh.j2: adapt to syntax change in Nautilus 14.2.17
+ (PR #589)
+- qa/common/json.sh: adapt to change in ceph status json format (PR #590)
+
## [1.11.0] - 2020-12-03
### Added
@@ -623,7 +647,8 @@
- Minimal README with a few usage instructions.
- The CHANGELOG file.
-[unreleased]: https://github.com/SUSE/sesdev/compare/v1.11.0...HEAD
+[unreleased]: https://github.com/SUSE/sesdev/compare/v1.12.0...HEAD
+[1.12.0]: https://github.com/SUSE/sesdev/releases/tag/v1.12.0
[1.11.0]: https://github.com/SUSE/sesdev/releases/tag/v1.11.0
[1.10.0]: https://github.com/SUSE/sesdev/releases/tag/v1.10.0
[1.9.1]: https://github.com/SUSE/sesdev/releases/tag/v1.9.1
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/sesdev-1.11.0+1607012263.g73dea97/README.md
new/sesdev-1.12.0+1616368506.g9bba186/README.md
--- old/sesdev-1.11.0+1607012263.g73dea97/README.md 2020-12-03
17:17:43.807857359 +0100
+++ new/sesdev-1.12.0+1616368506.g9bba186/README.md 2021-03-22
00:15:06.135033744 +0100
@@ -318,12 +318,9 @@
Second, make sure your virtualenv is active (`source venv/bin/activate`).
-Third, execute the following command in the top-level of your local git clone
-to install the dependencies, including test dependencies:
-
-```
-pip3 --use-feature=2020-resolver install --editable "./[dev]"
-```
+At this point, run `tox --version` to check if tox is already installed on your
+system. If it is not, then run `pip3 install tox` to install it in the Python
+virtual environment.
Finally, inspect the list of testing environments in `tox.ini` and choose one
or
more that you are interested in. Here is an example, but the actual output
might
@@ -331,25 +328,21 @@
```
$ tox --listenvs
-py36
+py3
lint
```
-(This means you have three testing environments to choose from: `py36`, `py37`,
-and `lint`.)
+(This means you have two testing environments to choose from: `py3` and
`lint`.)
Finally, run your chosen test environment(s):
```
-tox -e py36
+tox -e py3
tox -e lint
```
If you don't know which testing environment to choose, the command `tox` will
-run *all* the testing environments.
-
-CAVEAT: environments like `py36` will only run if that exact version of Python
-is installed on your system.
+run _all_ the testing environments.
## Usage
@@ -381,7 +374,7 @@
* `admin` - signifying that the node should get ceph.conf and keyring [1]
* `bootstrap` - The node where `cephadm bootstrap` will be run
* `client` - Various Ceph client utilities
-* `nfs` - NFS (Ganesha) gateway [2]
+* `nfs` - NFS (Ganesha) gateway [2] [5]
* `grafana` - Grafana metrics visualization (requires Prometheus) [3]
* `igw` - iSCSI target gateway
* `mds` - CephFS MDS
@@ -408,6 +401,8 @@
[Rook and CaaSP based Ceph cluster](#rook-and-caasp-based-ceph-cluster) for
more
information.
+[5] Not currently supported by the `octopus`, `pacific`, or `master` roles.
+
The following example will generate a cluster with four nodes: the master (Salt
Master) node that is also running a MON daemon; a storage (OSD) node that
will also run a MON, a MGR and an MDS and serve as the bootstrap node; another
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/sesdev-1.11.0+1607012263.g73dea97/contrib/upgrade-demo-ses6-to-ses7.sh
new/sesdev-1.12.0+1616368506.g9bba186/contrib/upgrade-demo-ses6-to-ses7.sh
--- old/sesdev-1.11.0+1607012263.g73dea97/contrib/upgrade-demo-ses6-to-ses7.sh
2020-12-03 17:17:43.807857359 +0100
+++ new/sesdev-1.12.0+1616368506.g9bba186/contrib/upgrade-demo-ses6-to-ses7.sh
2021-03-22 00:15:06.135033744 +0100
@@ -72,7 +72,15 @@
sesdev ssh "$DEP_ID" master salt-run upgrade.status
# TODO: parse output of upgrade.status and compare actual with expected output
sesdev ssh "$DEP_ID" master ceph health
-# TODO: assert HEALTH_OK
+
+function assert_health_ok {
+ if sesdev ssh ses6-to-ses7 master ceph health | grep HEALTH_OK ; then
+ echo "=> HEALTH_OK confirmed"
+ else
+ echo "ERROR: cluster not in HEALTH_OK!"
+ false
+ fi
+}
function ping_salt_minion {
set +x
@@ -105,6 +113,14 @@
function upgrade_node {
local node_short_hostname="$1"
+
+ echo
+ echo "=> BEGIN: upgrade $node_short_hostname"
+
+ echo
+ echo "=> set noout for OSDs on $node_short_hostname"
+ sesdev ssh "$DEP_ID" master ceph osd add-noout "$node_short_hostname"
+
echo
echo "=> upgrade $node_short_hostname of the Ceph cluster to
SLE-15-SP2/SES7 RPMs" > /dev/null
sesdev upgrade "$DEP_ID" "$node_short_hostname" --to ses7
@@ -126,37 +142,26 @@
# TODO: compare actual with expected output
sesdev ssh "$DEP_ID" master ceph health
# TODO: compare actual with expected output (probably won't be HEALTH_OK)
+ # TODO: based on the node's roles, assert that mons, mgrs, osds, etc. are
+ # really on octopus code
+
+ echo
+ echo "=> remove noout for $node_short_hostname"
+ sesdev ssh "$DEP_ID" master ceph osd rm-noout "$node_short_hostname"
+
+ echo
+ echo "=> END: upgrade $node_short_hostname"
}
echo
-echo "=> BEGIN: upgrade node1"
-sesdev ssh "$DEP_ID" master ceph osd add-noout node1
-# TODO: compare actual with expected output
+echo "=> assert HEALTH_OK after Salt Master node upgrade complete"
+assert_health_ok
+
upgrade_node node1
-sesdev ssh "$DEP_ID" master ceph osd rm-noout node1
-# TODO: compare actual with expected output
-echo
-echo "=> END: upgrade node1"
-echo
-echo "=> BEGIN: upgrade node2"
-sesdev ssh "$DEP_ID" master ceph osd add-noout node2
-# TODO: compare actual with expected output
upgrade_node node2
-sesdev ssh "$DEP_ID" master ceph osd rm-noout node2
-# TODO: compare actual with expected output
-echo
-echo "=> END: upgrade node2"
-echo
-echo "=> BEGIN: upgrade node3"
-sesdev ssh "$DEP_ID" master ceph osd add-noout node3
-# TODO: compare actual with expected output
upgrade_node node3
-sesdev ssh "$DEP_ID" master ceph osd rm-noout node3
-# TODO: compare actual with expected output
-echo
-echo "=> END: upgrade node3"
echo
echo "=> fully upgraded SES7 cluster" > /dev/null
@@ -180,7 +185,8 @@
echo "=> import and apply ceph-salt config" > /dev/null
sesdev ssh "$DEP_ID" master "bash -x -c 'ceph-salt import
ceph-salt-config.json'"
sesdev ssh "$DEP_ID" master "bash -x -c 'ceph-salt config /ssh generate'"
+#sesdev ssh "$DEP_ID" master "bash -x -c 'ceph-salt config
/ceph_cluster/roles/bootstrap reset'"
sesdev ssh "$DEP_ID" master "bash -x -c 'ceph-salt config ls'"
# TODO: compare actual with expected output
-sesdev ssh "$DEP_ID" master "bash -x -c 'ceph-salt apply'"
+sesdev ssh "$DEP_ID" master "bash -x -c 'ceph-salt apply --non-interactive'"
# TODO: validation (?)
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/sesdev-1.11.0+1607012263.g73dea97/qa/common/common.sh
new/sesdev-1.12.0+1616368506.g9bba186/qa/common/common.sh
--- old/sesdev-1.11.0+1607012263.g73dea97/qa/common/common.sh 2020-12-03
17:17:43.807857359 +0100
+++ new/sesdev-1.12.0+1616368506.g9bba186/qa/common/common.sh 2021-03-22
00:15:06.135033744 +0100
@@ -101,6 +101,57 @@
# core validation tests
#
+function assert_reboot_not_needed {
+ echo
+ echo "WWWW: assert_reboot_not_needed"
+ local success
+ success="yes"
+ local nodes_arr
+ local node_count
+ node_count="0"
+ local node_under_test
+ local n
+ local IFS
+ IFS=","
+ read -r -a nodes_arr <<<"$NODE_LIST"
+ for (( n=0; n<${#nodes_arr[*]}; n++ )) ; do
+ node_under_test="${nodes_arr[n]}"
+ node_under_test="${node_under_test//[$'\t\r\n']}"
+ node_count="$((node_count + 1))"
+ if ssh "$node_under_test" zypper ps -s | grep 'No processes using
deleted files found' ; then
+ success="yes"
+ else
+ set +e -x
+ zypper ps -s
+ hostname --fqdn
+ set +x -e
+ success=""
+ break
+ fi
+ done
+ echo
+ echo "WWWW: assert_reboot_not_needed: Checked $node_count of
${#nodes_arr[*]} nodes"
+ echo
+ if [ "$node_count" = "${#nodes_arr[*]}" ] ; then
+ true
+ else
+ success=""
+ fi
+ if [ "$success" ] ; then
+ echo "WWWW: assert_reboot_not_needed: OK"
+ echo
+ else
+ echo "WARNING: Running processes using deleted files detected!"
+ echo
+ echo "Since the presence of running processes using deleted files can"
+ echo "skew the test results, it would be better to reboot the cluster"
+ echo "before running qa-test."
+ echo
+ echo "WWWW: assert_reboot_not_needed: WARN"
+ echo
+ fi
+}
+
function support_cop_out_test {
set +x
local supported
@@ -1303,3 +1354,59 @@
function node_exporter_smoke_test {
_monitoring_smoke_test "node-exporter" "title.Node.Exporter"
}
+
+function core_dump_test {
+ local skipped
+ local succeeded
+ local osd_0_ps
+ local osd_0_pid
+ echo "WWWW: core_dump_test"
+ if [ "$VERSION_ID" = "15.2" ] || [ "$ID" = "opensuse-tumbleweed" ] ; then
+ echo "Asserting that there are no existing coredumps in the system."
+ set -x
+ coredumpctl list 2>&1 | grep 'No coredumps found'
+ set +x
+ echo "Looking for osd.0 on this node"
+ set -x
+ osd_0_ps="$(pgrep -a ceph-osd | grep osd\.0 | xargs)"
+ set +x
+ if [ "$osd_0_ps" ] ; then
+ echo "Found \"pgrep\" line corresponding to osd.0:"
+ echo "$osd_0_ps"
+ osd_0_pid="${osd_0_ps%% *}"
+ if [ "$osd_0_pid" ] ; then
+ echo "Extracted PID of osd.0: $osd_0_pid"
+ echo "Sending SIGSEGV to process $osd_0_pid"
+ set -x
+ kill -SEGV "$osd_0_pid"
+ sleep 30
+ coredumpctl list
+ set +x
+ if coredumpctl list 2>&1 | grep 'No coredumps found' ; then
+ echo "ERROR: no core dump collected"
+ else
+ succeeded="yes"
+ fi
+ else
+ echo "ERROR: could not extract PID from \"pgrep\" line?"
+ fi
+ else
+ echo "osd.0 does not appear to be running on this node."
+ skipped="yes"
+ fi
+ else
+ echo "Test only works on recent SUSE OSes."
+ skipped="yes"
+ fi
+ if [ "$skipped" ] ; then
+ echo "core_dump_test: SKIPPED"
+ echo
+ elif [ "$succeeded" ] ; then
+ echo "core_dump_test: OK"
+ echo
+ else
+ echo "core_dump_test: FAIL"
+ echo
+ false
+ fi
+}
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/sesdev-1.11.0+1607012263.g73dea97/qa/common/json.sh
new/sesdev-1.12.0+1616368506.g9bba186/qa/common/json.sh
--- old/sesdev-1.11.0+1607012263.g73dea97/qa/common/json.sh 2020-12-03
17:17:43.807857359 +0100
+++ new/sesdev-1.12.0+1616368506.g9bba186/qa/common/json.sh 2021-03-22
00:15:06.135033744 +0100
@@ -90,8 +90,12 @@
local ceph_status_json
if [ "$VERSION_ID" = "15.2" ] || [ "$ID" = "opensuse-tumbleweed" ] ; then
json_ses7_orch_ls mon
- elif [ "$VERSION_ID" = "15.1" ] || [ "$VERSION_ID" = "12.3" ] ; then
- # SES6, SES5
+ elif [ "$VERSION_ID" = "15.1" ] ; then
+ # SES6
+ ceph_status_json="$(ceph status --format json)"
+ echo "$ceph_status_json" | jq -r ".monmap.num_mons"
+ elif [ "$VERSION_ID" = "12.3" ] ; then
+ # SES5
ceph_status_json="$(ceph status --format json)"
echo "$ceph_status_json" | jq -r ".monmap.mons | length"
else
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/sesdev-1.11.0+1607012263.g73dea97/qa/health-ok.sh
new/sesdev-1.12.0+1616368506.g9bba186/qa/health-ok.sh
--- old/sesdev-1.11.0+1607012263.g73dea97/qa/health-ok.sh 2020-12-03
17:17:43.807857359 +0100
+++ new/sesdev-1.12.0+1616368506.g9bba186/qa/health-ok.sh 2021-03-22
00:15:06.135033744 +0100
@@ -178,6 +178,7 @@
set -e
# tests that verify basic assumptions
+assert_reboot_not_needed
support_cop_out_test
no_non_oss_repos_test
make_salt_master_an_admin_node_test
@@ -198,6 +199,9 @@
# check that OSDs have the expected objectstore
osd_objectstore_test
+# core dump test
+core_dump_test
+
# check numbers of daemons and whether they are running on the expected nodes
number_of_daemons_expected_vs_metadata_test
number_of_services_expected_vs_orch_ls_test
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/sesdev-1.11.0+1607012263.g73dea97/sesdev/__init__.py
new/sesdev-1.12.0+1616368506.g9bba186/sesdev/__init__.py
--- old/sesdev-1.11.0+1607012263.g73dea97/sesdev/__init__.py 2020-12-03
17:17:43.807857359 +0100
+++ new/sesdev-1.12.0+1616368506.g9bba186/sesdev/__init__.py 2021-03-22
00:15:06.135033744 +0100
@@ -239,6 +239,14 @@
return dep_id
+def _maybe_munge_repo_url(repo_url):
+ retval = repo_url
+ match = re.search(r'(http.+\/).+\.repo$', repo_url)
+ if match:
+ retval = match.group(1)
+ return retval
+
+
@click.group()
@click.option('-w', '--work-path', required=False,
type=click.Path(exists=True, dir_okay=True, file_okay=False),
@@ -404,7 +412,9 @@
raise NoExplicitRolesWithSingleNode()
elif single_node:
roles_string = ""
- if version in ['ses7', 'octopus', 'pacific']:
+ if version in ['ses7']:
+ roles_string = Constant.ROLES_SINGLE_NODE['ses7']
+ elif version in ['octopus', 'pacific']:
roles_string = Constant.ROLES_SINGLE_NODE['octopus']
elif version in ['ses6', 'nautilus']:
roles_string = Constant.ROLES_SINGLE_NODE['nautilus']
@@ -481,6 +491,8 @@
if deepsea_branch is not None:
settings_dict['deepsea_git_branch'] = deepsea_branch
+ if not deepsea_repo:
+ settings_dict['deepsea_git_repo'] = Constant.DEEPSEA_REPO
if version is not None:
settings_dict['version'] = version
@@ -501,7 +513,7 @@
settings_dict['custom_repos'].append(
{
'name': 'custom-repo-{}'.format(count),
- 'url': repo_url,
+ 'url': _maybe_munge_repo_url(repo_url),
'priority': Constant.ZYPPER_PRIO_ELEVATED if repo_priority
else None
})
@@ -602,18 +614,25 @@
for folder in synced_folder:
try:
src, dst = folder.split(':')
- if not all([path.isabs(x) for x in [src, dst]]):
- raise OptionValueError('--synced-folder',
- "Please provide absolute paths for "
- "synced folder paths",
- folder)
- if not path.exists(src):
- raise OptionValueError('--synced-folder',
- "Path to the source synced folder must
exist",
- src)
-
+ if path.isabs(src) and path.isabs(dst):
+ pass
+ else:
+ raise OptionValueError(
+ "--synced-folder",
+ "Please provide absolute paths for synced folder paths",
+ folder
+ )
+ if path.exists(src):
+ pass
+ else:
+ raise OptionValueError(
+ "--synced-folder",
+ "Path to the source synced folder must exist",
+ src
+ )
except ValueError as exc:
raise OptionFormatError('--synced-folder', "src:dst", folder) from
exc
+
settings_dict['synced_folder'] = [folder.split(':') for folder in
synced_folder]
if msgr2_secure_mode:
@@ -920,7 +939,7 @@
if kwargs['custom_repo']:
custom_repo = ZypperRepo(
name='custom_repo_{}'.format(tools.gen_random_string(6)),
- url=kwargs['custom_repo'],
+ url=_maybe_munge_repo_url(kwargs['custom_repo']),
priority=Constant.ZYPPER_PRIO_ELEVATED if kwargs['repo_priority']
else None
)
dep.add_repo_subcommand(custom_repo, kwargs['update'], _print_log)
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/sesdev-1.11.0+1607012263.g73dea97/sesdev.spec
new/sesdev-1.12.0+1616368506.g9bba186/sesdev.spec
--- old/sesdev-1.11.0+1607012263.g73dea97/sesdev.spec 2020-12-03
17:17:43.995858850 +0100
+++ new/sesdev-1.12.0+1616368506.g9bba186/sesdev.spec 2021-03-22
00:15:06.431034550 +0100
@@ -16,7 +16,7 @@
#
Name: sesdev
-Version: 1.11.0+1607012263.g73dea97
+Version: 1.12.0+1616368506.g9bba186
Release: 1%{?dist}
Summary: CLI tool to deploy and manage SES clusters
License: MIT
@@ -92,6 +92,7 @@
install -m 0644 qa/common/common.sh
%{buildroot}/%{_datadir}/%{name}/qa/common/common.sh
install -m 0644 qa/common/helper.sh
%{buildroot}/%{_datadir}/%{name}/qa/common/helper.sh
install -m 0644 qa/common/json.sh
%{buildroot}/%{_datadir}/%{name}/qa/common/json.sh
+install -m 0644 qa/common/rgw.sh
%{buildroot}/%{_datadir}/%{name}/qa/common/rgw.sh
install -m 0644 qa/common/zypper.sh
%{buildroot}/%{_datadir}/%{name}/qa/common/zypper.sh
%files
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/sesdev-1.11.0+1607012263.g73dea97/seslib/constant.py
new/sesdev-1.12.0+1616368506.g9bba186/seslib/constant.py
--- old/sesdev-1.11.0+1607012263.g73dea97/seslib/constant.py 2020-12-03
17:17:43.807857359 +0100
+++ new/sesdev-1.12.0+1616368506.g9bba186/seslib/constant.py 2021-03-22
00:15:06.135033744 +0100
@@ -18,6 +18,8 @@
DEBUG = False
+ DEEPSEA_REPO = "https://github.com/SUSE/DeepSea";
+
DEVELOPER_TOOLS_REPOS = {
'sles-15-sp1': {
'dev-tools': 'http://dist.suse.de/ibs/SUSE/Products/'
@@ -237,8 +239,13 @@
"octopus": [["admin", "master", "client", "prometheus", "grafana",
"alertmanager",
"node-exporter"],
["bootstrap", "storage", "mon", "mgr", "rgw", "igw",
"node-exporter"],
- ["storage", "mon", "mgr", "mds", "igw", "nfs",
"node-exporter"],
- ["storage", "mon", "mgr", "mds", "rgw", "nfs",
"node-exporter"]]
+ ["storage", "mon", "mgr", "mds", "igw", "node-exporter"],
+ ["storage", "mon", "mgr", "mds", "rgw", "node-exporter"]],
+ "ses7": [["admin", "master", "client", "prometheus", "grafana",
"alertmanager",
+ "node-exporter"],
+ ["bootstrap", "storage", "mon", "mgr", "rgw", "igw",
"node-exporter"],
+ ["storage", "mon", "mgr", "mds", "igw", "node-exporter"],
+ ["storage", "mon", "mgr", "mds", "rgw", "nfs",
"node-exporter"]]
}
ROLES_DEFAULT_BY_VERSION = {
@@ -249,7 +256,7 @@
'pacific': ROLES_DEFAULT["octopus"],
'ses5': ROLES_DEFAULT["luminous"],
'ses6': ROLES_DEFAULT["nautilus"],
- 'ses7': ROLES_DEFAULT["octopus"],
+ 'ses7': ROLES_DEFAULT["ses7"],
}
ROLES_KNOWN = [
@@ -282,7 +289,9 @@
"nautilus": "[ master, storage, mon, mgr, prometheus, grafana, mds,
igw, rgw, "
"nfs ]",
"octopus": "[ master, admin, bootstrap, storage, mon, mgr, mds, igw,
rgw, "
- "nfs, prometheus, grafana, alertmanager, node-exporter ]",
+ "prometheus, grafana, alertmanager, node-exporter ]",
+ "ses7": "[ master, admin, bootstrap, storage, mon, mgr, mds, igw, rgw,
"
+ "prometheus, grafana, alertmanager, node-exporter ]",
}
SSH_KEY_NAME = 'sesdev' # do NOT use 'id_rsa'
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/sesdev-1.11.0+1607012263.g73dea97/seslib/deployment.py
new/sesdev-1.12.0+1616368506.g9bba186/seslib/deployment.py
--- old/sesdev-1.11.0+1607012263.g73dea97/seslib/deployment.py 2020-12-03
17:17:43.807857359 +0100
+++ new/sesdev-1.12.0+1616368506.g9bba186/seslib/deployment.py 2021-03-22
00:15:06.135033744 +0100
@@ -514,6 +514,7 @@
'sesdev_path_to_qa': Constant.PATH_TO_QA,
'dep_id': self.dep_id,
'os': self.settings.os,
+ 'ram': self.settings.ram,
'package_manager':
Constant.OS_PACKAGE_MANAGER_MAPPING[self.settings.os],
'vm_engine': self.settings.vm_engine,
'libvirt_host': self.settings.libvirt_host,
@@ -950,6 +951,9 @@
result += "- FQDN: {}\n".format(self.settings.fqdn)
if self.settings.rgw_ssl:
result += "- RGW with SSL:
{}\n".format(self.settings.rgw_ssl)
+ if self.settings.deepsea_git_repo and
self.settings.deepsea_git_branch:
+ result += "- DeepSea repo:
{}\n".format(self.settings.deepsea_git_repo)
+ result += "- DeepSea branch:
{}\n".format(self.settings.deepsea_git_branch)
if show_individual_vms:
result += "\n"
result += "Individual VM parameters:\n"
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/sesdev-1.11.0+1607012263.g73dea97/seslib/templates/cephadm/deployment_day_2.sh.j2
new/sesdev-1.12.0+1616368506.g9bba186/seslib/templates/cephadm/deployment_day_2.sh.j2
---
old/sesdev-1.11.0+1607012263.g73dea97/seslib/templates/cephadm/deployment_day_2.sh.j2
2020-12-03 17:17:43.811857390 +0100
+++
new/sesdev-1.12.0+1616368506.g9bba186/seslib/templates/cephadm/deployment_day_2.sh.j2
2021-03-22 00:15:06.135033744 +0100
@@ -230,6 +230,7 @@
radosgw-admin realm create --rgw-realm=default --default
radosgw-admin zonegroup create --rgw-zonegroup=default --master --default
radosgw-admin zone create --rgw-zonegroup=default --rgw-zone=default --master
--default
+radosgw-admin period update --rgw-realm=default --commit
cat >> {{ service_spec_gw }} << 'EOF'
---
service_type: rgw
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/sesdev-1.11.0+1607012263.g73dea97/seslib/templates/provision.sh.j2
new/sesdev-1.12.0+1616368506.g9bba186/seslib/templates/provision.sh.j2
--- old/sesdev-1.11.0+1607012263.g73dea97/seslib/templates/provision.sh.j2
2020-12-03 17:17:43.811857390 +0100
+++ new/sesdev-1.12.0+1616368506.g9bba186/seslib/templates/provision.sh.j2
2021-03-22 00:15:06.139033754 +0100
@@ -16,6 +16,9 @@
# ensure Vagrant doesn't destroy the master node when the script fails
trap 'err_report $LINENO' ERR
+# do not limit coredump size
+ulimit -c unlimited
+
ls -lR /home/vagrant
# populate /etc/hosts
@@ -47,15 +50,11 @@
hostnamectl set-hostname {{ node.name }}
{% endif %}
+{% if version in ['octopus', 'ses7', 'pacific'] %}
# persist the journal
sed -i -e 's/#Storage=auto/Storage=persistent/' /etc/systemd/journald.conf
systemctl restart systemd-journald
-
-{% if not provision %}
-set +x
-echo "Stopping the deployment now because the --no-provision option was given"
-exit 0
-{% endif %}{# not provision #}
+{% endif %}{# version in ['octopus', 'ses7', 'pacific'] #}
# if --ssd option was given, set rotational flag on first additional disk
{% if ssd %}
@@ -74,6 +73,12 @@
{% include "dnf.j2" ignore missing %}
{% endif %}{# package_manager == 'zypper' #}
+{% if not provision %}
+set +x
+echo "Stopping the deployment now because the --no-provision option was given"
+exit 0
+{% endif %}{# not provision #}
+
# include helper scripts
{% include "helper_scripts.j2" %}
@@ -97,6 +102,23 @@
aa-status || true
{% endif %}{# apparmor #}
+{% if os != 'sles-12-sp3' %}
+source /etc/os-release
+if [[ "$ID_LIKE" =~ "suse" ]] ; then
+ # enable coredump capturing
+ sysctl -w kernel.core_pattern="|/usr/lib/systemd/systemd-coredump %P %u %g
s %t %c %e"
+ sysctl kernel.core_pattern
+ cat <<EOF >>/etc/systemd/coredump.conf
+ProcessSizeMax={{ ram }}G
+ExternalSizeMax={{ ram }}G
+JournalSizeMax={{ ram }}G
+EOF
+ cat /etc/systemd/coredump.conf
+ systemctl enable systemd-coredump.socket
+ systemctl start systemd-coredump.socket
+fi
+{% endif %}{# os != 'sles-12-sp3' #}
+
# deployment state machine
# DeepSea
@@ -134,3 +156,6 @@
{% include "ubuntu/provision.sh.j2" %}
{% endif %}{# end of deploy state machine #}
+
+# inform user if reboot is needed (should be the very last thing the
provisioner does)
+zypper ps -s || true
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/sesdev-1.11.0+1607012263.g73dea97/seslib/templates/salt/ceph-salt/deployment_day_1.sh.j2
new/sesdev-1.12.0+1616368506.g9bba186/seslib/templates/salt/ceph-salt/deployment_day_1.sh.j2
---
old/sesdev-1.11.0+1607012263.g73dea97/seslib/templates/salt/ceph-salt/deployment_day_1.sh.j2
2020-12-03 17:17:43.811857390 +0100
+++
new/sesdev-1.12.0+1616368506.g9bba186/seslib/templates/salt/ceph-salt/deployment_day_1.sh.j2
2021-03-22 00:15:06.139033754 +0100
@@ -49,7 +49,7 @@
ceph-salt --version
{% for node in nodes %}
-{% if not node.has_exclusive_role('client') %}
+{% if node.has_roles() and not node.has_exclusive_role('client') %}
ceph-salt config /ceph_cluster/minions add {{ node.fqdn }}
ceph-salt config /ceph_cluster/roles/cephadm add {{ node.fqdn }}
{% endif %}
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/sesdev-1.11.0+1607012263.g73dea97/seslib/templates/salt/deepsea/nautilus_pre_stage_4.sh.j2
new/sesdev-1.12.0+1616368506.g9bba186/seslib/templates/salt/deepsea/nautilus_pre_stage_4.sh.j2
---
old/sesdev-1.11.0+1607012263.g73dea97/seslib/templates/salt/deepsea/nautilus_pre_stage_4.sh.j2
2020-12-03 17:17:43.811857390 +0100
+++
new/sesdev-1.12.0+1616368506.g9bba186/seslib/templates/salt/deepsea/nautilus_pre_stage_4.sh.j2
2021-03-22 00:15:06.139033754 +0100
@@ -1,2 +1,2 @@
- # set dashboard password to 'admin'
- ceph dashboard ac-user-set-password admin admin
\ No newline at end of file
+# set password of dashboard user 'admin' to 'admin'
+echo -n admin | ceph dashboard ac-user-set-password admin -i -
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/sesdev-1.11.0+1607012263.g73dea97/seslib/templates/zypper.j2
new/sesdev-1.12.0+1616368506.g9bba186/seslib/templates/zypper.j2
--- old/sesdev-1.11.0+1607012263.g73dea97/seslib/templates/zypper.j2
2020-12-03 17:17:43.811857390 +0100
+++ new/sesdev-1.12.0+1616368506.g9bba186/seslib/templates/zypper.j2
2021-03-22 00:15:06.139033754 +0100
@@ -19,9 +19,8 @@
zypper --non-interactive remove which || true
# remove Python 2 so it doesn't pollute the environment
-{% if os != 'sles-12-sp3' %}
zypper --non-interactive remove python-base || true
-{% endif %}{# os != 'sles-12-sp3' #}
+zypper --non-interactive remove libpython2_7-1_0 || true
# remove Non-OSS repos in openSUSE
{% if os.startswith('leap') or os == "tumbleweed" %}
@@ -216,6 +215,7 @@
'bc',
'apparmor-utils',
'apparmor-parser',
+ 'lsof',
] %}
{% if os == 'sles-12-sp3' %}
zypper --non-interactive install {{ basic_pkgs_to_install | join(' ') }} ntp
@@ -223,6 +223,13 @@
zypper --non-interactive install {{ basic_pkgs_to_install | join(' ') }}
chrony hostname
{% endif %}{# os == 'sles-12-sp3' #}
+{% if os != 'sles-12-sp3' %}
+# install systemd-coredump, ensuring that it has the same exact version as
systemd
+# (avoid newer systemd possibly pre-installed by Kiwi in the Vagrant Box)
+zypper --non-interactive install --force systemd systemd-sysvinit udev libudev1
+zypper --non-interactive install systemd-coredump
+{% endif %}{# os != 'sles-12-sp3' #}
+
{% if os.startswith("sle") %}
{% set sle_pkgs_to_install = [
'ca-certificates-suse',
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/sesdev-1.11.0+1607012263.g73dea97/tox.ini
new/sesdev-1.12.0+1616368506.g9bba186/tox.ini
--- old/sesdev-1.11.0+1607012263.g73dea97/tox.ini 2020-12-03
17:17:43.811857390 +0100
+++ new/sesdev-1.12.0+1616368506.g9bba186/tox.ini 2021-03-22
00:15:06.139033754 +0100
@@ -1,6 +1,6 @@
[tox]
-envlist = py36,lint
-minversion = 2.4
+envlist = py3,lint
+minversion = 3.4
skipsdist = True
basepython = python3
@@ -8,12 +8,15 @@
ignore = E123,E124,E126,E129,W503,W504
[testenv]
+deps =
+ click
+ pytest
+ pytest-pycodestyle
+ pylint
usedevelop = True
-install_command = pip3 --use-feature=2020-resolver install {opts} {packages}
-extras = dev
commands =
- pytest {posargs: -vv}
- pytest --pycodestyle {posargs: -vv}
+ pytest {posargs: -vv}
+ pytest --pycodestyle {posargs: -vv}
[testenv:lint]
allowlist_externals = pylint
