Script 'mail_helper' called by obssrc
Hello community,
here is the log from the commit of package transactional-update for
openSUSE:Factory checked in at 2021-08-05 20:47:53
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Comparing /work/SRC/openSUSE:Factory/transactional-update (Old)
and /work/SRC/openSUSE:Factory/.transactional-update.new.1899 (New)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "transactional-update"
Thu Aug 5 20:47:53 2021 rev:74 rq:909993 version:3.5.1
Changes:
--------
---
/work/SRC/openSUSE:Factory/transactional-update/transactional-update.changes
2021-06-16 20:33:25.350974952 +0200
+++
/work/SRC/openSUSE:Factory/.transactional-update.new.1899/transactional-update.changes
2021-08-05 20:48:13.583923733 +0200
@@ -1,0 +2,32 @@
+Tue Aug 3 12:41:41 UTC 2021 - Ignaz Forster <[email protected]>
+
+- Version 3.5.1
+ - t-u: Disable status file generation by default
+ The new experimental `status` command requires the availability of
+ /etc/YaST2/control.xml, which is not present on all systems. Hide the
+ creation of the corresponding status file behind a new EXPERIMENTAL_STATUS
+ option to try out this functionality.
+ - Increase library version
+
+-------------------------------------------------------------------
+Tue Aug 3 07:37:36 UTC 2021 - Ignaz Forster <[email protected]>
+
+- Add tukit.conf to spec file
+
+-------------------------------------------------------------------
+Mon Aug 2 15:58:09 UTC 2021 - Ignaz Forster <[email protected]>
+
+- Version 3.5.0
+ - Add alias setDiscardIfUnchanged for setDiscard. The old method name
+ wasn't really clear and will be removed if we should have an API break
+ in the future
+ - Replace mkinitrd with direct dracut call [boo#1186213]
+ - tukit: Add configuration file support (/etc/tukit.conf)
+ - Allow users to configure additional bind mounts (see /usr/etc/tukit.conf
+ for an example and limitations) [bsc#1188322]
+ - Add 'transactional-update status' call. This is a POC for obtaining a
+ hash of a system to verify its integrity. The functionality is still
+ experimental!
+ - Internal bugfixes / optimizations
+
+-------------------------------------------------------------------
Old:
----
transactional-update-3.4.0.tar.gz
New:
----
transactional-update-3.5.1.tar.gz
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Other differences:
------------------
++++++ transactional-update.spec ++++++
--- /var/tmp/diff_new_pack.GXIzTi/_old 2021-08-05 20:48:14.079923176 +0200
+++ /var/tmp/diff_new_pack.GXIzTi/_new 2021-08-05 20:48:14.083923172 +0200
@@ -26,7 +26,7 @@
%{!?_distconfdir: %global _distconfdir %{_prefix}%{_sysconfdir}}
Name: transactional-update
-Version: 3.4.0
+Version: 3.5.1
Release: 0
Summary: Transactional Updates with btrfs and snapshots
License: GPL-2.0-or-later AND LGPL-2.1-or-later
@@ -212,6 +212,7 @@
%dir %{_distconfdir}
%endif
%{_distconfdir}/transactional-update.conf
+%{_distconfdir}/tukit.conf
%{_mandir}/man5/transactional-update.conf.5*
%{_mandir}/man8/transactional-update.8*
%{_mandir}/man8/transactional-update.timer.8*
++++++ transactional-update-3.4.0.tar.gz -> transactional-update-3.5.1.tar.gz
++++++
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/transactional-update-3.4.0/NEWS
new/transactional-update-3.5.1/NEWS
--- old/transactional-update-3.4.0/NEWS 2021-06-14 00:37:42.000000000 +0200
+++ new/transactional-update-3.5.1/NEWS 2021-08-03 14:34:57.000000000 +0200
@@ -2,6 +2,37 @@
Copyright (C) 2016-2020 Thorsten Kukuk, Ignaz Forster et al.
+Version 3.5.1
+* t-u: Disable status file generation by default
+ The new experimental `status` command requires the availability of
+ /etc/YaST2/control.xml, which is not present on all systems. Hide the
+ creation of the corresponding status file behind a new EXPERIMENTAL_STATUS
+ option to try out this functionality.
+* Increase library version
+
+Version 3.5.0
+* Add alias setDiscardIfUnchanged for setDiscard. The old method name
+ wasn't really clear and will be removed if we should have an API break
+ in the future
+* Replace mkinitrd with direct dracut call [boo#1186213]
+* tukit: Add configuration file support (/etc/tukit.conf)
+* Allow users to configure additional bind mounts (see /usr/etc/tukit.conf
+ for an example and limitations) [bsc#1188322]
+* Add 'transactional-update status' call. This is a POC for obtaining a
+ hash of a system to verify its integrity. The functionality is still
+ experimental!
+* Internal bugfixes / optimizations
+
+Version 3.4.0
+* Apply SElinux context on /etc in transaction [boo#1185625], [boo#1185766]
+ [bsc#1186842], [boo#1186775]
+* Implement inotify handling in C instead of Bash; this makes the
+ --drop-if-no-change option work on SLE Micro [bsc#1184529]
+* Use `tukit call` for up, dup and patch to allow resuming an update after
+ zypper updated itself in the snapshot [bsc#1185226]
+* Fix obsolete output type messages in initrd [boo#1177149]
+* Make different base snapshot warning more visible [bsc#1185224]
+
Version 3.3.0
* Add support for more package managers by bind mounting their directories
* Support snapshots without dedicated overlay [boo#1183539], [bsc#1183539]
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/transactional-update-3.4.0/configure.ac
new/transactional-update-3.5.1/configure.ac
--- old/transactional-update-3.4.0/configure.ac 2021-06-14 00:37:42.000000000
+0200
+++ new/transactional-update-3.5.1/configure.ac 2021-08-03 14:34:57.000000000
+0200
@@ -1,11 +1,11 @@
dnl Process this file with autoconf to produce a configure script.
-AC_INIT(transactional-update, 3.3.0)
+AC_INIT(transactional-update, 3.5.1)
# Increase on any interface change and reset revision
-LIBTOOL_CURRENT=2
+LIBTOOL_CURRENT=3
# Increase or reset on any VERSION update
-LIBTOOL_REVISION=2
+LIBTOOL_REVISION=1
# Increase if interface change is backwards compatible, reset otherwise
-LIBTOOL_AGE=2
+LIBTOOL_AGE=3
AC_CANONICAL_SYSTEM
AM_INIT_AUTOMAKE([foreign])
AC_CONFIG_FILES([tukit.pc])
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/transactional-update-3.4.0/etc/Makefile.am
new/transactional-update-3.5.1/etc/Makefile.am
--- old/transactional-update-3.4.0/etc/Makefile.am 2021-06-14
00:37:42.000000000 +0200
+++ new/transactional-update-3.5.1/etc/Makefile.am 2021-08-03
14:34:57.000000000 +0200
@@ -1,12 +1,6 @@
-#
-# Copyright (c) 2018 Ignaz Forster <[email protected]>
-#
+# SPDX-License-Identifier: LGPL-2.1-or-later
+# SPDX-FileCopyrightText: 2018-2021 SUSE LLC
-EXTRA_DIST = transactional-update.conf
-
-install-data-local:
- $(mkinstalldirs) $(DESTDIR)$(prefix)$(sysconfdir)
- $(INSTALL_DATA) $(srcdir)/$(EXTRA_DIST) $(DESTDIR)$(prefix)$(sysconfdir)
-
-uninstall-local:
- -rm $(DESTDIR)$(prefix)$(sysconfdir)/$(EXTRA_DIST)
+EXTRA_DIST = transactional-update.conf tukit.conf
+configdir = $(prefix)$(sysconfdir)
+config_DATA = $(EXTRA_DIST)
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/transactional-update-3.4.0/etc/tukit.conf
new/transactional-update-3.5.1/etc/tukit.conf
--- old/transactional-update-3.4.0/etc/tukit.conf 1970-01-01
01:00:00.000000000 +0100
+++ new/transactional-update-3.5.1/etc/tukit.conf 2021-08-03
14:34:57.000000000 +0200
@@ -0,0 +1,13 @@
+# Bind mount additional directories into the update environment; to add
+# multiple directories add a separate entry with a different id between the
+# brackets for each.
+# Be aware that transactions are only done the root file system itself, i.e.
+# any additional mounts added here will *not* get a seperate snapshot (so
+# their contents cannot be rolled back) and the changes will be visible to
+# the running system immediately (breaking the transactional concept and
+# possibly resulting in a mix between old and new files if there are also
+# changes to the root file system until the next reboot).
+# By default only /opt will be bind-mounted, as this directory is often
+# used by third-party packages, but the applications in there are usually
+# self-contained.
+BINDDIRS[0]="/opt"
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/transactional-update-3.4.0/lib/Configuration.cpp
new/transactional-update-3.5.1/lib/Configuration.cpp
--- old/transactional-update-3.4.0/lib/Configuration.cpp 2021-06-14
00:37:42.000000000 +0200
+++ new/transactional-update-3.5.1/lib/Configuration.cpp 2021-08-03
14:34:57.000000000 +0200
@@ -1,5 +1,5 @@
/* SPDX-License-Identifier: LGPL-2.1-or-later */
-/* SPDX-FileCopyrightText: 2020 SUSE LLC */
+/* SPDX-FileCopyrightText: 2020-2021 SUSE LLC */
/*
Retrieves configuration values, set via configuration file or using
@@ -9,13 +9,15 @@
#include "Configuration.hpp"
#include "Util.hpp"
#include <map>
+#include <regex>
#include <stdexcept>
#include <libeconf.h>
namespace TransactionalUpdate {
Configuration::Configuration() {
- econf_err error = econf_newIniFile(&key_file);
+ econf_file *kf_defaults;
+ econf_err error = econf_newIniFile(&kf_defaults);
if (error)
throw std::runtime_error{"Could not create default configuration."};
std::map<const char*, const char*> defaults = {
@@ -24,9 +26,31 @@
{"OVERLAY_DIR", "/var/lib/overlay"}
};
for(auto &[key, value] : defaults) {
- error = econf_setStringValue(key_file, "", key, value);
- if (error)
+ error = econf_setStringValue(kf_defaults, "", key, value);
+ if (error) {
+ econf_freeFile(kf_defaults);
throw std::runtime_error{"Could not set default value for '" +
std::string(key) + "'."};
+ }
+ }
+
+ econf_file *kf_conffiles;
+ error = econf_readDirs(&kf_conffiles, (std::string(PREFIX) +
CONFDIR).c_str(), CONFDIR, "tukit", ".conf", "=", "#");
+ if (error && error != ECONF_NOFILE) {
+ econf_freeFile(kf_defaults);
+ econf_freeFile(kf_conffiles);
+ throw std::runtime_error{"Couldn't read configuration file: " +
std::string(econf_errString(error))};
+ }
+
+ if (error == ECONF_SUCCESS) {
+ error = econf_mergeFiles(&key_file, kf_defaults, kf_conffiles);
+ econf_freeFile(kf_defaults);
+ econf_freeFile(kf_conffiles);
+ if (error) {
+ throw std::runtime_error{"Couldn't merge configuration: " +
std::string(econf_errString(error))};
+ }
+ } else {
+ econf_freeFile(key_file);
+ key_file = std::move(kf_defaults);
}
}
@@ -42,4 +66,32 @@
return std::string(val);
}
+std::vector<std::string> Configuration::getArray(const std::string &key) {
+ std::vector<std::string> ret;
+ econf_err error;
+
+ size_t len = 0;
+ char** confkeys;
+ error = econf_getKeys(key_file, "", &len, &confkeys);
+ if (error)
+ throw std::runtime_error{"Could not read keys."};
+
+ std::regex exp(key + "\\[.*\\]");
+ for (size_t i = 0; i < len; i++) {
+ if (std::regex_match(confkeys[i], exp)) {
+ CString val;
+ error = econf_getStringValue(key_file, "", confkeys[i], &val.ptr);
+ if (error) {
+ econf_freeArray(confkeys);
+ throw std::runtime_error{"Could not read key '" +
std::string(confkeys[i]) + "'."};
+ }
+ if (val != nullptr)
+ ret.push_back(std::string(val));
+ }
+ }
+ econf_freeArray(confkeys);
+
+ return ret;
+}
+
} // namespace TransactionalUpdate
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/transactional-update-3.4.0/lib/Configuration.hpp
new/transactional-update-3.5.1/lib/Configuration.hpp
--- old/transactional-update-3.4.0/lib/Configuration.hpp 2021-06-14
00:37:42.000000000 +0200
+++ new/transactional-update-3.5.1/lib/Configuration.hpp 2021-08-03
14:34:57.000000000 +0200
@@ -10,6 +10,7 @@
#define T_U_CONFIGURATION_H
#include <string>
+#include <vector>
typedef struct econf_file econf_file;
@@ -22,6 +23,7 @@
Configuration(const Configuration&) = delete;
void operator=(const Configuration&) = delete;
std::string get(const std::string &key);
+ std::vector<std::string> getArray(const std::string &key);
private:
econf_file *key_file;
};
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/transactional-update-3.4.0/lib/Makefile.am
new/transactional-update-3.5.1/lib/Makefile.am
--- old/transactional-update-3.4.0/lib/Makefile.am 2021-06-14
00:37:42.000000000 +0200
+++ new/transactional-update-3.5.1/lib/Makefile.am 2021-08-03
14:34:57.000000000 +0200
@@ -9,6 +9,6 @@
noinst_HEADERS=Snapshot.hpp Snapshot/Snapper.hpp \
Mount.hpp Overlay.hpp Log.hpp Configuration.hpp \
Util.hpp Supplement.hpp Exceptions.hpp
-libtukit_la_CPPFLAGS=$(ECONF_CFLAGS) $(LIBMOUNT_CFLAGS) $(SELINUX_CFLAGS)
+libtukit_la_CPPFLAGS=-DPREFIX=\"$(prefix)\" -DCONFDIR=\"$(sysconfdir)\"
$(ECONF_CFLAGS) $(LIBMOUNT_CFLAGS) $(SELINUX_CFLAGS)
libtukit_la_LDFLAGS=$(ECONF_LIBS) $(LIBMOUNT_LIBS) $(SELINUX_LIBS) \
-version-info $(LIBTOOL_CURRENT):$(LIBTOOL_REVISION):$(LIBTOOL_AGE)
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/transactional-update-3.4.0/lib/Transaction.cpp
new/transactional-update-3.5.1/lib/Transaction.cpp
--- old/transactional-update-3.4.0/lib/Transaction.cpp 2021-06-14
00:37:42.000000000 +0200
+++ new/transactional-update-3.5.1/lib/Transaction.cpp 2021-08-03
14:34:57.000000000 +0200
@@ -20,7 +20,6 @@
#include <cstdlib>
#include <cstring>
#include <fstream>
-#include <future>
#include <ftw.h>
#include <limits.h>
#include <poll.h>
@@ -40,13 +39,12 @@
void mount();
int runCommand(char* argv[], bool inChroot);
static int inotifyAdd(const char *pathname, const struct stat *sbuf, int
type, struct FTW *ftwb);
- static void inotifyRead();
+ int inotifyRead();
std::unique_ptr<Snapshot> snapshot;
std::string bindDir;
std::vector<std::unique_ptr<Mount>> dirsToMount;
Supplements supplements;
pid_t pidCmd;
- std::future<void> inotifyListener;
bool discardIfNoChange = false;
};
@@ -97,7 +95,11 @@
void Transaction::impl::mount() {
dirsToMount.push_back(std::make_unique<PropagatedBindMount>("/dev"));
dirsToMount.push_back(std::make_unique<BindMount>("/var/log"));
- dirsToMount.push_back(std::make_unique<BindMount>("/opt"));
+
+ std::vector<std::string> customDirs = config.getArray("BINDDIRS");
+ for (auto it = customDirs.begin(); it != customDirs.end(); ++it) {
+ dirsToMount.push_back(std::make_unique<BindMount>(*it));
+ }
Mount mntVar{"/var"};
if (mntVar.isMount()) {
@@ -235,18 +237,21 @@
}
}
-void Transaction::setDiscard(bool discard) {
+void Transaction::setDiscardIfUnchanged(bool discard) {
pImpl->discardIfNoChange = discard;
}
+void Transaction::setDiscard(bool discard) {
+ setDiscardIfUnchanged(discard);
+}
-void Transaction::impl::inotifyRead() {
+int Transaction::impl::inotifyRead() {
size_t bufLen = sizeof(struct inotify_event) + NAME_MAX + 1;
char buf[bufLen] __attribute__((aligned(8)));
ssize_t numRead;
int ret;
struct pollfd pfd = {inotifyFd, POLLIN, 0};
- while ((ret = (poll(&pfd, 1, 500))) == 0) {}
+ ret = (poll(&pfd, 1, 500));
if (ret == -1) {
throw std::runtime_error{"Polling inotify file descriptior failed: " +
std::string(strerror(errno))};
} else if (ret > 0) {
@@ -254,9 +259,10 @@
if (numRead == 0)
throw std::runtime_error{"Read() from inotify fd returned 0!"};
if (numRead == -1)
- throw std::runtime_error{"read"};
+ throw std::runtime_error{"Reading from inotify fd failed: " +
std::string(strerror(errno))};
tulog.debug("inotify: Exiting after event on ", ((struct inotify_event
*)buf)->name);
}
+ return ret;
}
int Transaction::impl::runCommand(char* argv[], bool inChroot) {
@@ -267,8 +273,6 @@
// Recursively register all directories of the root file system
nftw(snapshot->getRoot().c_str(), inotifyAdd, 20, FTW_MOUNT |
FTW_PHYS);
-
- inotifyListener = std::async(inotifyRead);
}
std::string opts = "Executing `";
@@ -302,6 +306,7 @@
if (execvp(argv[0], (char* const*)argv) < 0) {
throw std::runtime_error{"Calling " + std::string(argv[0]) + "
failed: " + std::string(strerror(errno))};
}
+ ret = -1;
} else {
this->pidCmd = pid;
ret = waitpid(pid, &status, 0);
@@ -347,7 +352,7 @@
void Transaction::finalize() {
sync();
if (pImpl->discardIfNoChange &&
- ((inotifyFd != 0 &&
pImpl->inotifyListener.wait_for(std::chrono::milliseconds(0)) !=
std::future_status::ready) ||
+ ((inotifyFd != 0 && pImpl->inotifyRead() == 0) ||
(inotifyFd == 0 && fs::exists(pImpl->snapshot->getRoot() /
"discardIfNoChange")))) {
tulog.info("No changes to the root file system - discarding
snapshot.");
@@ -383,7 +388,7 @@
void Transaction::keep() {
sync();
- if (fs::exists(pImpl->snapshot->getRoot() / "discardIfNoChange") &&
(inotifyFd != 0 &&
pImpl->inotifyListener.wait_for(std::chrono::milliseconds(0)) ==
std::future_status::ready)) {
+ if (fs::exists(pImpl->snapshot->getRoot() / "discardIfNoChange") &&
(inotifyFd != 0 && pImpl->inotifyRead() > 0)) {
tulog.debug("Snapshot was changed, removing discard flagfile.");
fs::remove(pImpl->snapshot->getRoot() / "discardIfNoChange");
}
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/transactional-update-3.4.0/lib/Transaction.hpp
new/transactional-update-3.5.1/lib/Transaction.hpp
--- old/transactional-update-3.4.0/lib/Transaction.hpp 2021-06-14
00:37:42.000000000 +0200
+++ new/transactional-update-3.5.1/lib/Transaction.hpp 2021-08-03
14:34:57.000000000 +0200
@@ -68,6 +68,8 @@
* Be aware that inotify registration may fail, e.g. if a system has a lot
of open inotify
* listeners already. Changes may not be detected correctly in this case.
*/
+ void setDiscardIfUnchanged(bool discard);
+ /** Temporary legacy name - remove on the next incompatible interface
change. **/
void setDiscard(bool discard);
/**
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/transactional-update-3.4.0/man/transactional-update.8.xml
new/transactional-update-3.5.1/man/transactional-update.8.xml
--- old/transactional-update-3.4.0/man/transactional-update.8.xml
2021-06-14 00:37:42.000000000 +0200
+++ new/transactional-update-3.5.1/man/transactional-update.8.xml
2021-08-03 14:34:57.000000000 +0200
@@ -441,6 +441,64 @@
</para>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>status</option></term>
+ <listitem>
+ <warning>
+ This command is under development and will change in the
+ future. The status command is currently using a non-stable
+ feature of MicroOS, the control.xml file, that will change the
+ scope and location in future releases.
+ </warning>
+ <para>
+ Shows the status of current and older snapshots. The status
+ try to represent how the system diverges from an ideal base
+ image, and the list of patterns and packages added and removed
+ from it.
+ </para>
+ <para>
+ The status is represented via a hash (currently sha256) of a
+ manifest file that contains the list of expected packages and
+ the build time of those packages. The list of expected
+ packages are calculated using the <command>libsolv</command>
+ library, and resolving it for the list of default patterns
+ associated with the role used during the installation.
+ </para>
+ <para>
+ Two manifest files are created. One will use the information
+ from the repositories used for the installation. The list of
+ packages (including the full version and architecture) and the
+ build time will be extracted from the repository information.
+ The other one will use the same list of packages, but using
+ the build time from the current system (if the package is
+ present).
+ </para>
+ <para>
+ Both manifest files will be the same if we have a fresh
+ installed system. But in distributions like MicroOS, that are
+ released based on a rolling model, the subsequent updates will
+ make compromises that will result in a diverged state. In that
+ way, if two systems have the same BASE_MANIFEST_DIGEST (the
+ digest calculated from the repository information), we can
+ assume that are in the same state even if the
+ SYSTEM_MANIFEST_DIGEST is different.
+ </para>
+ <para>
+ We can require of two system to be in the exact same status,
+ and for that we can compare both digests.
+ </para>
+ <para>
+ The status also shows the differences in patterns and packages
+ from the expected situation, and will save a copy of both
+ manifest for further audit of the system.
+ </para>
+ <para>
+ To write the necessary state file
+ <option>EXPERIMENTAL_STATUS=1</option>has to be set in
+ <filename>transactional-update.conf</filename>.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
</refsect2>
</refsect1>
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/transactional-update-3.4.0/sbin/transactional-update.in
new/transactional-update-3.5.1/sbin/transactional-update.in
--- old/transactional-update-3.4.0/sbin/transactional-update.in 2021-06-14
00:37:42.000000000 +0200
+++ new/transactional-update-3.5.1/sbin/transactional-update.in 2021-08-03
14:34:57.000000000 +0200
@@ -38,6 +38,8 @@
DO_SELF_UPDATE=1
DO_REGISTRATION=0
DO_RUN=0
+DO_STATUS=0
+DO_STATUS_LAST=0
REGISTRATION_ARGS=""
ROLLBACK_SNAPSHOT=0
REBOOT_AFTERWARDS=0
@@ -55,6 +57,10 @@
NEW_SNAPSHOT_FLAG="/var/lib/overlay/transactional-update.newsnapshot"
NEEDS_RESTARTING_FILE="/var/run/reboot-needed"
LOCKFILE="/var/run/transactional-update.pid"
+STATUS_FILE="@libdir@/sysimage/tu/transactional-update.status"
+EXPERIMENTAL_STATUS=0
+BASE_MANIFEST_FILE="@libdir@/sysimage/tu/base.manifest"
+SYSTEM_MANIFEST_FILE="@libdir@/sysimage/tu/system.manifest"
ZYPPER_AUTO_IMPORT_KEYS=0
ETC_OVERLAY_PATTERN='^[^[:space:]]\+[[:space:]]\+\/etc[[:space:]]\+overlay[[:space:]]\+\([^[:space:]]*,\|\)workdir=\/sysroot\/var\/lib\/overlay\/work-etc[,[:space:]]'
NON_ROOTFS_WHITELIST=("/var/lib/YaST2/cookies" "/var/lib/rpm"
"/var/lib/systemd/migrated" "/var/run/zypp.pid")
@@ -80,6 +86,10 @@
TMPFILE=""
DO_CALLEXT=0
+declare -A ROLES=()
+REPOS=()
+declare -A BUILDTIME=()
+
# Create stderr alias for things that shouldn't be logged into logfile
if [ ! -e /proc/$$/fd/4 ]; then
exec 4>&2
@@ -152,6 +162,7 @@
echo "Standalone Commands:"
echo "rollback [<number>] Set the current or given snapshot as
default snapshot"
echo "rollback last Set the last working snapshot as default
snapshot"
+ echo "status Status and history report [experimental -
enabled: ${EXPERIMENTAL_STATUS}]"
echo ""
echo "Options:"
echo "--interactive, -i Use interactive mode for package command"
@@ -177,6 +188,10 @@
fi
}
+log_to_file() {
+ echo -e `date "+%Y-%m-%d %H:%M"` "$@" >> ${LOGFILE}
+}
+
log_error() {
TELEM_PAYLOAD="${TELEM_PAYLOAD}\nmessage=$@"
echo -e `date "+%Y-%m-%d %H:%M"` "$@" >> ${LOGFILE}
@@ -349,6 +364,401 @@
touch "${VARDIR}/check-registration"
}
+# Return the optimized list of mount points
+root_mount_points() {
+ local prev=0
+ for mount in $(findmnt --noheadings --submounts --target / --output TARGET
--raw | sort | tail -n +2); do
+ if [ "${prev}/" != "${mount:0:$((${#prev}+1))}" ]; then
+ echo "${mount}"
+ prev="${mount}"
+ fi
+ done
+}
+
+# Call tukit callext with the correct parameters
+callext() {
+ tukit ${TUKIT_OPTS} --quiet callext "${SNAPSHOT_ID}" "$@"
+}
+
+# Helper function to grep perl regular expressions
+re_findall() {
+ grep --only-matching --perl-regexp "$1" "${2:--}"
+}
+
+# XXX bsc#1170709 can impact here, but is not expected tha the base
+# product change.
+# Get the baseproduct name
+baseproduct() {
+ local file="/etc/products.d/baseproduct"
+ callext rpm --root {} --quiet --query --file "$file" || log_to_file "File
$file not owned: check bsc#1170709"
+
+ re_findall '(?<=<name>).*?(?=</name>)' "${SNAPSHOT_DIR}${file}"
+}
+
+# XXX bsc#1170709 will impact a lot here. We need a more consistent
+# proposal.
+# Read all the roles from control.xml and store in a global variable
+load_roles() {
+ local file="/etc/YaST2/control.xml"
+ callext rpm --root {} --quiet --query --file "$file" || log_to_file "File
$file not owned: check bsc#1170709"
+
+ # TODO: detect the microos_selinux pattern
+
+ # Parsing control.xml cannot be done directly by grep, as the
+ # logic of default_patterns can be a bit complex in the case of
+ # MicroOS. We can have some general default_patterns, and roles
+ # without default_patterns.
+ local xpath="//*[local-name()='productDefines']"
+ xpath+="/*[local-name()='software']"
+ xpath+="/*[local-name()='default_patterns']/text()"
+
+ local default_patterns=$(xmllint --xpath "$xpath" "${SNAPSHOT_DIR}${file}")
+
+ xpath="//*[local-name()='system_role']"
+ xpath+="/*[local-name()='id']/text()"
+ local role patterns
+ while read role; do
+ xpath="//*[local-name()='system_role'][*[local-name()='id']='"$role"']"
+ xpath+="/*[local-name()='software']"
+ xpath+="/*[local-name()='default_patterns']/text()"
+ patterns=$(xmllint --xpath "$xpath" "${SNAPSHOT_DIR}${file}" 2>
/dev/null)
+ ROLES["$role"]="${patterns:-$default_patterns}"
+ done <<< $(xmllint --xpath "$xpath" "${SNAPSHOT_DIR}${file}")
+}
+
+# Read all the repo aliases and the priority, reverse sorted by
+# priority
+load_repo_alias() {
+ local regex='(?<=alias=").*?(?=")'
+ regex+='|(?<=priority=").*?(?=")'
+ regex+='|(?<=enabled=").*?(?=")'
+ regex+='|(?<=<url>).*?(?=</url>)'
+
+ # Get the repositories, remove the ones that have duplicated URL,
+ # and sort by priority
+ local line
+ while read line; do
+ line=($line)
+ [ "${line[2]}" != "0" ] && REPOS+=("${line[0]} ${line[1]}")
+ done <<< $(callext zypper --root {} --xmlout lr | re_findall "$regex" |
xargs -n4 | LC_COLLATE=C sort -k4 | uniq -f3 | LC_COLLATE=C sort -k2 -r)
+}
+
+# Read all the build time of repo packages and store in a file
+load_buildtime_for_repo() {
+ local file="/var/cache/zypp/raw/$1/repodata/*primary.xml.gz"
+ local regex='(?<=<name>).*(?=</name>)'
+ regex+='|(?<=<arch>).*(?=</arch>)'
+ regex+='|(?<=<version ).*?(?=/>)'
+ regex+='|(?<=<time ).*?(?=/>)'
+
+ local name arch line epoch ver rel file build
+ while read name; do
+ read arch
+ read line; eval "$line"
+ read line; eval "$line"
+ [ ! -z "$name" ] && echo "${name}-${ver}-${rel}.${arch} $build" >> "$2"
+ done <<< $(zcat $file | re_findall "$regex")
+}
+
+# Read all the build time of installed packages
+load_buildtime() {
+ local line
+ while read line; do
+ line=($line)
+ BUILDTIME["${line[0]}"]="${line[1]}"
+ done <<< $(callext rpm --root {} --query --all --queryformat='%{NVRA}
%{BUILDTIME}\n')
+}
+
+# Return the timestamp of a repository from the metadata
+timestamp_for_repo() {
+ local file="/var/cache/zypp/raw/$1/repodata/repomd.xml"
+ re_findall '(?<=<revision>).*?(?=</revision>)' "$file"
+}
+
+# Return 0 if Zypper will ignore recommended packages
+only_requires() {
+ local file="/etc/zypp/zypp.conf"
+ callext grep --quiet '^solver.onlyRequires\s*=\s*true'
"${SNAPSHOT_DIR}${file}"
+ return #?
+}
+
+# Return 0 if Zypper allow vendor change
+allow_vendor_change() {
+ local file="/etc/zypp/zypp.conf"
+ callext grep --quiet '^solver.allowVendorChange\s*=\s*true'
"${SNAPSHOT_DIR}${file}"
+ return #?
+}
+
+# Create a libsolv testcase for a role
+testcase_for_role() {
+ local arch="$(uname -m)"
+
+ echo "system $arch rpm"
+
+ local repo
+ for repo in "${REPOS[@]}"; do
+ repo=($repo)
+ echo "repo ${repo[0]} ${repo[1]} solv
/var/cache/zypp/solv/${repo[0]}/solv"
+ done
+
+ local flags=""
+ only_requires && flags="ignorerecommended"
+ allow_vendor_change && flags+=" allowvendorchange"
+ [ ! -z "$flags" ] && echo "solverflags $flags"
+
+ echo "job install name product:$(baseproduct)"
+
+ local role
+ for role in ${ROLES[$1]}; do
+ echo "job install name pattern:$role"
+ done
+}
+
+# Given a role, calculate the list of all packages and store it in a
+# file
+load_packages_for_role() {
+ local testcase_file="$(mktemp)"
+
+ testcase_for_role "$1" > "$testcase_file"
+
+ # Execute in a subshell because of a testsolv bug, that do not
+ # accept full path names
+ (
+ cd $(dirname "$testcase_file")
+ testsolv $(basename "$testcase_file") | re_findall '(?<= - ).*' |
LC_COLLATE=C sort > "$2"
+ )
+
+ rm -f "$testcase_file"
+}
+
+# Calculate the manifest document using the repository information
+base_manifest() {
+ local pkg
+ while read pkg; do
+ grep "^$pkg " "$1" || echo "$pkg"
+ done <"$2"
+}
+
+# Calculate the manifest document from the current system
+system_manifest() {
+ local pkg
+ while read pkg; do
+ if [ "${BUILDTIME["$pkg"]+_}" ]; then
+ echo "$pkg ${BUILDTIME["$pkg"]}"
+ else
+ echo "$pkg"
+ fi
+ done <"$1"
+}
+
+# List the installed patterns by the user
+installed_patterns() {
+ callext zypper --root {} --disable-repositories se --installed-only
--sort-by-name --type pattern \
+ | re_findall '(?<=i\+ \| ).*?(?= )'
+}
+
+# List all the installed patterns (including hiddens one)
+all_installed_patterns() {
+ callext rpm --root {} --query --provides --whatprovides 'pattern()' \
+ | re_findall '(?<=pattern\(\) = ).*' \
+ | LC_COLLATE=C sort
+}
+
+# List the installed packages by the user
+installed_packages() {
+ callext zypper --root {} --disable-repositories se --installed-only
--sort-by-name --type package \
+ | re_findall '(?<=i\+ \| ).*?(?= )'
+}
+
+# List all installed packages
+all_installed_packages() {
+ callext rpm --root {} --query --all --queryformat='%{NAME}\n' \
+ | LC_COLLATE=C sort
+}
+
+# List the extra patterns installed by the user
+added_patterns() {
+ # TODO The selinux one should be detected
+ local exclude='microos_selinux'
+ local inst_patterns="$(installed_patterns | grep --invert-match
--extended-regexp "$exclude")"
+ local role_patterns="$(echo "${ROLES[$1]}" | tr ' ' '\n' | LC_COLLATE=C
sort)"
+
+ LC_COLLATE=C comm -23 <(echo "${inst_patterns}") <(echo "${role_patterns}")
+}
+
+# List the patterns from the role that are not installed
+removed_patterns() {
+ local inst_patterns="$(all_installed_patterns)"
+ local role_patterns="$(echo "${ROLES[$1]}" | tr ' ' '\n' | LC_COLLATE=C
sort)"
+
+ LC_COLLATE=C comm -23 <(echo "${role_patterns}") <(echo "${inst_patterns}")
+}
+
+# List the extra packages installed by the user
+added_packages() {
+ # TODO What others are installed by YaST?
+ # TODO Maybe add them into packages_for_role_extended, or add them
+ # in the configuration file
+ local exclude='patterns-|kernel-|snapper|grub2|btrfsprogs'
+ local inst_packages="$(installed_packages | grep --invert-match
--extended-regexp "$exclude")"
+ local role_packages="$(cat "$1" | rev | cut -d"-" -f3- | rev |
LC_COLLATE=C sort)"
+
+ LC_COLLATE=C comm -23 <(echo "${inst_packages}") <(echo "${role_packages}")
+}
+
+# List the packages from the role that are not installed
+removed_packages() {
+ local exclude='pattern:|product:'
+ local inst_packages="$(all_installed_packages)"
+ local role_packages="$(cat "$1" | grep --invert-match --extended-regexp
"$exclude" | rev | cut -d"-" -f3- | rev | LC_COLLATE=C sort)"
+
+ LC_COLLATE=C comm -23 <(echo "${role_packages}") <(echo "${inst_packages}")
+}
+
+# Calculate the Jaccard index for two sets
+jaccard() {
+ local union_card=$(echo "$1 $2" | tr ' ' '\n' | LC_COLLATE=C sort | uniq |
wc -l)
+ local intersection_card=$(echo "$1 $2" | tr ' ' '\n' | LC_COLLATE=C sort |
uniq -D | uniq | wc -l)
+ echo $((100 * $intersection_card / $union_card))
+}
+
+# Based on the installed patterns, find the installed role
+find_closer_role() {
+ local patterns="$(installed_patterns)"
+ local max_index=0
+ local max_role=
+ local index=
+
+ for role in "${!ROLES[@]}"; do
+ index="$(jaccard "$patterns" "${ROLES["$role"]}")"
+ if [ "$index" -gt "$max_index" ]; then
+ max_index="$index"
+ max_role="$role"
+ fi
+ done
+
+ echo "$max_role"
+}
+
+# Wait for a series of PIDs
+wait_all() {
+ local pid
+ for pid in "$@"; do
+ wait "$pid"
+ done
+}
+
+# Create the status file and all the assets to validate it in the
+# future by the user
+create_status_file() {
+ # Start from a clean state
+ rm -fr "$(dirname ${SNAPSHOT_DIR}${STATUS_FILE})"
+ mkdir -p "$(dirname ${SNAPSHOT_DIR}${STATUS_FILE})"
+
+ echo "DATE=\"$(date +"%Y-%m-%d %T")\"" >> "${SNAPSHOT_DIR}${STATUS_FILE}"
+ echo "PRODUCT=\"$(baseproduct)\"" >> "${SNAPSHOT_DIR}${STATUS_FILE}"
+ grep VERSION_ID "${SNAPSHOT_DIR}/etc/os-release" >>
"${SNAPSHOT_DIR}${STATUS_FILE}"
+
+ load_roles
+ local role="$(find_closer_role)"
+ echo "ROLE=\"$role\"" >> "${SNAPSHOT_DIR}${STATUS_FILE}"
+
+ load_repo_alias
+
+ local repo_buildtime_file="$(mktemp)"
+ local repo
+ local pids=()
+ for repo in "${REPOS[@]}"; do
+ repo=($repo)
+ log_info "Parsing repo ${repo[0]}"
+ # Do the load in the background, and store the data in the
+ # file. The OS will take care of synchronizing the file
+ # access (or at least seems so)
+ load_buildtime_for_repo "${repo[0]}" "$repo_buildtime_file" & pids+=($!)
+ done
+ load_buildtime
+ wait_all "${pids[@]}"
+
+ local packages_for_role_file="$(mktemp)"
+ load_packages_for_role "$role" "$packages_for_role_file"
+
+ base_manifest "$repo_buildtime_file" "$packages_for_role_file" >
"${SNAPSHOT_DIR}${BASE_MANIFEST_FILE}"
+ system_manifest "$packages_for_role_file" >
"${SNAPSHOT_DIR}${SYSTEM_MANIFEST_FILE}"
+
+ local base_manifest_digest="$(sha256sum
"${SNAPSHOT_DIR}${BASE_MANIFEST_FILE}" | cut -d' ' -f1)"
+ local system_manifest_digest="$(sha256sum
"${SNAPSHOT_DIR}${SYSTEM_MANIFEST_FILE}" | cut -d' ' -f1)"
+
+ echo "BASE_MANIFEST_DIGEST=\"$base_manifest_digest\"" >>
"${SNAPSHOT_DIR}${STATUS_FILE}"
+ echo "SYSTEM_MANIFEST_DIGEST=\"$system_manifest_digest\"" >>
"${SNAPSHOT_DIR}${STATUS_FILE}"
+
+ pids=()
+ gzip "${SNAPSHOT_DIR}${BASE_MANIFEST_FILE}" & pids+=($!)
+ gzip "${SNAPSHOT_DIR}${SYSTEM_MANIFEST_FILE}" & pids+=($!)
+
+ local items=$(added_patterns "$role" | xargs)
+ echo "ADDED_PATTERNS=\"$items\"" >> "${SNAPSHOT_DIR}${STATUS_FILE}"
+ items=$(removed_patterns "$role" | xargs)
+ echo "REMOVED_PATTERNS=\"$items\"" >> "${SNAPSHOT_DIR}${STATUS_FILE}"
+
+ items=$(added_packages "$packages_for_role_file" | xargs)
+ echo "ADDED_PACKAGES=\"$items\"" >> "${SNAPSHOT_DIR}${STATUS_FILE}"
+ items=$(removed_packages "$packages_for_role_file" | xargs)
+ echo "REMOVED_PACKAGES=\"$items\"" >> "${SNAPSHOT_DIR}${STATUS_FILE}"
+
+ wait_all "${pids[@]}"
+
+ rm -f "$repo_buildtime_file" "$packages_for_role_file"
+}
+
+# Get the snapshot number
+snapshot_num() {
+ local regex='(?<=<num>).*?(?=</num>)'
+ re_findall "$regex" "${1}info.xml"
+}
+
+# Get the snapshot date, in the same format that the status file
+snapshot_date() {
+ local regex='(?<=<date>).*?(?=</date>)'
+ re_findall "$regex" "${1}info.xml"
+}
+
+# Get the snapshot description line
+snapshot_description() {
+ local regex='(?<=<description>).*?(?=</description>)'
+ re_findall "$regex" "${1}info.xml"
+}
+
+# Show the status of a snapshot
+show_snapshot_status() {
+ printf "%0.s-" $(seq 1 $(tput cols)); echo
+
+ local ss_id=$(snapshot_num "$1")
+ if [ "$CURRENT_SNAPSHOT_ID" -eq "$ss_id" ]; then
+ ss_id+="*"
+ elif [ "$DEFAULT_SNAPSHOT_ID" -eq "$ss_id" ]; then
+ ss_id+="+"
+ fi
+
+ local ss_date="$(snapshot_date "$1")"
+ local ss_description="$(snapshot_description "$1")"
+ echo "Snapshot: ${ss_id} [$(snapshot_date "$1")]"
+ echo "Description: $(snapshot_description "$1")"
+ if [ -f "${1}snapshot${STATUS_FILE}" ]; then
+ . "${1}snapshot${STATUS_FILE}"
+ echo "Report date: ${DATE}"
+ echo "Version: ${PRODUCT} ${VERSION_ID}"
+ echo "Base OS hash: ${BASE_MANIFEST_DIGEST}"
+ echo "Effective OS hash: ${SYSTEM_MANIFEST_DIGEST}"
+ [ ! -z "${ADDED_PATTERNS}" ] && echo "Added patterns:
${ADDED_PATTERNS}"
+ [ ! -z "${REMOVED_PATTERNS}" ] && echo "Removed patterns:
${REMOVED_PATTERNS}"
+ [ ! -z "${ADDED_PACKAGES}" ] && echo "Added packages:
${ADDED_PACKAGES}"
+ [ ! -z "${REMOVED_PACKAGES}" ] && echo "Removed packages:
${REMOVED_PACKAGES}"
+ else
+ echo "(Missing status file)"
+ fi
+}
+
+
ORIG_ARGS=("$@")
while [ 1 ]; do
@@ -511,6 +921,23 @@
SETUP_SELINUX=1
shift
;;
+ status)
+ DO_STATUS=1
+ DO_SELF_UPDATE=0
+ shift
+ while [ 1 ]; do
+ [ $# -eq 0 ] && break
+ case "$1" in
+ last)
+ DO_STATUS_LAST=1
+ shift
+ ;;
+ *)
+ usage 1;
+ ;;
+ esac
+ done
+ ;;
-i|--interactive)
ZYPPER_NONINTERACTIVE=""
shift
@@ -752,6 +1179,14 @@
exit 0
fi
+if [ "${DO_STATUS}" -eq 1 ]; then
+ for snapshot in $(ls -d /.snapshots/*/ | sort --reverse --numeric-sort); do
+ show_snapshot_status "$snapshot"
+ [ "${DO_STATUS_LAST}" -eq 1 ] && break
+ done
+ exit 0
+fi
+
#
# Cleanup part: make sure old root file systems will be removed after they are
no longer active.
#
@@ -912,6 +1347,11 @@
fi
fi
+ # Create the status for the new snapshot, before closing it
+ if [ "${EXPERIMENTAL_STATUS}" -eq 1 ]; then
+ create_status_file
+ fi
+
if [ $RETVAL -eq 0 -o $RETVAL -eq 102 -o $RETVAL -eq 103 -o \( $DO_DUP
-eq 0 -a $RETVAL -eq 106 \) ]; then
REBUILD_KDUMP_INITRD=1
# check if products are updated and we need to re-register
@@ -965,9 +1405,9 @@
if [ ${REWRITE_INITRD} -eq 1 ]; then
log_info "Creating new initrd"
- tukit ${TUKIT_OPTS} call "${SNAPSHOT_ID}" /sbin/mkinitrd
+ tukit ${TUKIT_OPTS} call "${SNAPSHOT_ID}" dracut --force
--regenerate-all
if [ $? -ne 0 ]; then
- log_error "ERROR: mkinitrd failed!"
+ log_error "ERROR: initrd creation failed!"
EXITCODE=1
else
REBUILD_KDUMP_INITRD=1
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/transactional-update-3.4.0/tukit/Makefile.am
new/transactional-update-3.5.1/tukit/Makefile.am
--- old/transactional-update-3.4.0/tukit/Makefile.am 2021-06-14
00:37:42.000000000 +0200
+++ new/transactional-update-3.5.1/tukit/Makefile.am 2021-08-03
14:34:57.000000000 +0200
@@ -4,4 +4,4 @@
tukit.cpp
noinst_HEADERS=tukit.hpp
tukit_CPPFLAGS = -I $(top_srcdir)/lib $(ECONF_CFLAGS)
-tukit_LDADD = $(top_builddir)/lib/libtukit.la $(ECONF_LIBS) -lmount -lpthread
+tukit_LDADD = $(top_builddir)/lib/libtukit.la $(ECONF_LIBS) -lmount
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/transactional-update-3.4.0/tukit/tukit.cpp
new/transactional-update-3.5.1/tukit/tukit.cpp
--- old/transactional-update-3.4.0/tukit/tukit.cpp 2021-06-14
00:37:42.000000000 +0200
+++ new/transactional-update-3.5.1/tukit/tukit.cpp 2021-08-03
14:34:57.000000000 +0200
@@ -112,7 +112,7 @@
string arg = argv[0];
if (arg == "execute") {
if (discardSnapshot) {
- transaction.setDiscard(true);
+ transaction.setDiscardIfUnchanged(true);
}
transaction.init(baseSnapshot);
int status = transaction.execute(&argv[1]); // All remaining arguments
@@ -125,7 +125,7 @@
}
else if (arg == "open") {
if (discardSnapshot) {
- transaction.setDiscard(true);
+ transaction.setDiscardIfUnchanged(true);
}
transaction.init(baseSnapshot);
cout << "ID: " << transaction.getSnapshot() << endl;
