Bug#801156: dpkg: sometimes does not pass old version to postinst on upgrade

[Andreas Beckmann]

On 2015-10-10 16:13, Guillem Jover wrote:
> Thanks for the recipe. I was going through the code to see why this
> might be happening and realized that it is most probably caused by
> systemd being in triggers-pending state, which the code was not taking
> into account. I've prepared a patch, but I've not yet setup the test
> environment, if you have it ready, could you try to test a dpkg with
> this patch to see if it fixed it for you?

Yes, this seems to fix the issue.

(I didn't manage to build dpkg from git with git-buildpackage and
pbuilder, so I ended up patching the 1.18.3 tarball and pbuilding that)


Andreas

Bug#801156: dpkg: sometimes does not pass old version to postinst on upgrade

[Guillem Jover]

Hi!

On Sat, 2015-10-10 at 16:13:11 +0200, Guillem Jover wrote:
> Thanks for the recipe. I was going through the code to see why this
> might be happening and realized that it is most probably caused by
> systemd being in triggers-pending state, which the code was not taking
> into account. I've prepared a patch, but I've not yet setup the test
> environment, if you have it ready, could you try to test a dpkg with
> this patch to see if it fixed it for you?

> I think I might instead create a reduced test case for the dpkg-tests
> functional test suite, which should be faster to reproduce and retest.

I've done this now, and I can reproduce it, and the previously attached
patch fixes the issue. I'll push the fix later today, and queue it for
all the supported stable releases. The test case is:

  


Knowing that it also fixes the issues in piuparts would be valuable I
guess, but I'd be very happy if someone else could check that out. :)

Thanks,
Guillem

Bug#801156: dpkg: sometimes does not pass old version to postinst on upgrade

[Guillem Jover]

Hi!

On Wed, 2015-10-07 at 03:29:10 +0200, Andreas Beckmann wrote:
> The chroot was bootstrapped with
> 
> 'debootstrap' '--variant=minbase' 
> '--keyring=/usr/share/keyrings/debian-archive-keyring.gpg' 
> '--components=main' '--arch=amd64' 'jessie' '/tmp/piupartss/tmpcNOJI3' 
> 'http://ftp.de.debian.org/debian'
> 
> and was slightly configured by piuparts to have e.g.
> 
> # cat /etc/apt/apt.conf.d/piuparts 
> APT::Get::Assume-Yes "yes";
> APT::Install-Recommends "0";
> APT::Install-Suggests "0";
> APT::Get::AllowUnauthenticated "No";
> Acquire::PDiffs "false";
> Acquire::http::Proxy "http://localhost:3128";;
> Dpkg::Options {"--force-unsafe-io";};

Thanks for the recipe. I was going through the code to see why this
might be happening and realized that it is most probably caused by
systemd being in triggers-pending state, which the code was not taking
into account. I've prepared a patch, but I've not yet setup the test
environment, if you have it ready, could you try to test a dpkg with
this patch to see if it fixed it for you?

I think I might instead create a reduced test case for the dpkg-tests
functional test suite, which should be faster to reproduce and retest.

Thanks,
Guillem
From 61a5fbc39deb116eec20db7757cf0a00ac8d5a0d Mon Sep 17 00:00:00 2001
From: Guillem Jover 
Date: Sat, 10 Oct 2015 16:06:41 +0200
Subject: [PATCH] libdpkg: Config-Version should also be initialized on
 triggers-pending

A package in triggers-pending state should be considered an installed
package, by not doing so we might end up not passing the correct version
to the configure maintainer script and making it look like we are doing
a configuration for a first install, instead of an upgrade.

Closes: #801156
Reported-by: Andreas Beckmann 
Stable-Candidate: 1.16.x 1.17.x
---
 lib/dpkg/parse.c | 19 ---
 1 file changed, 12 insertions(+), 7 deletions(-)

diff --git a/lib/dpkg/parse.c b/lib/dpkg/parse.c
index 043a164..c06626a 100644
--- a/lib/dpkg/parse.c
+++ b/lib/dpkg/parse.c
@@ -225,19 +225,24 @@ pkg_parse_verify(struct parsedb_state *ps,
   if (!dop->arch)
 dop->arch = pkgbin->arch;
 
-  /* Check the Config-Version information:
-   * If there is a Config-Version it is definitely to be used, but
-   * there shouldn't be one if the package is ‘installed’ (in which case
-   * the Version and/or Revision will be copied) or if the package is
-   * ‘not-installed’ (in which case there is no Config-Version). */
+  /*
+   * Check the Config-Version information:
+   *
+   * If there is a Config-Version it is definitely to be used, but there
+   * should not be one if the package is ‘installed’ or ‘triggers-pending’
+   * (in which case the Version and/or Revision will be copied) or if the
+   * package is ‘not-installed’ (in which case there is no Config-Version).
+   */
   if (!(ps->flags & pdb_recordavailable)) {
 if (pkg->configversion.version) {
   if (pkg->status == PKG_STAT_INSTALLED ||
-  pkg->status == PKG_STAT_NOTINSTALLED)
+  pkg->status == PKG_STAT_NOTINSTALLED ||
+  pkg->status == PKG_STAT_TRIGGERSPENDING)
 parse_error(ps,
 _("Config-Version for package with inappropriate Status"));
 } else {
-  if (pkg->status == PKG_STAT_INSTALLED)
+  if (pkg->status == PKG_STAT_INSTALLED ||
+  pkg->status == PKG_STAT_TRIGGERSPENDING)
 pkg->configversion = pkgbin->version;
 }
   }
-- 
2.6.1

Bug#801156: dpkg: sometimes does not pass old version to postinst on upgrade

[Andreas Beckmann]

The chroot was bootstrapped with

'debootstrap' '--variant=minbase' 
'--keyring=/usr/share/keyrings/debian-archive-keyring.gpg' '--components=main' 
'--arch=amd64' 'jessie' '/tmp/piupartss/tmpcNOJI3' 
'http://ftp.de.debian.org/debian'

and was slightly configured by piuparts to have e.g.

# cat /etc/apt/apt.conf.d/piuparts 
APT::Get::Assume-Yes "yes";
APT::Install-Recommends "0";
APT::Install-Suggests "0";
APT::Get::AllowUnauthenticated "No";
Acquire::PDiffs "false";
Acquire::http::Proxy "http://localhost:3128";;
Dpkg::Options {"--force-unsafe-io";};


Andreas

Bug#801156: dpkg: sometimes does not pass old version to postinst on upgrade

[Andreas Beckmann]

On 2015-10-07 02:23, Guillem Jover wrote:
> Hi!
> 
> On Wed, 2015-10-07 at 01:10:33 +0200, Andreas Beckmann wrote:
>> Package: dpkg
>> Version: 1.18.3
>> Severity: serious
> 
>> dpkg does not always properly pass the old version number when
>> calling "postinst configure" on package upgrades, sometimes there
>> is either no or an empty argument.
> 
> That's to be expected, the second argument is the last version that
> got configured, if no version has ever been configured then the
> argument is omitted. Checking now, this is documented in Debian
> policy §6.7.

> I've not checked the logs, but I'd assume that in the problematic case
> the package has never been configured before, and as such the bug here
> is on that specific package.

That is unlikely. Both tests start from the same chroot tarball
(debootstrapped about a month ago)

Right before packing it up the status was listed as

  ii  systemd   215-17+deb8u2amd64system and service manager

so I assume it has been configured during the debootstrap run.

If I enter that chroot with pbuilder login, the status of the systemd
package is listed as

Package: systemd
Status: install ok installed
Priority: important
Section: admin
Installed-Size: 13838
Maintainer: Debian systemd Maintainers

Architecture: amd64
Version: 215-17+deb8u2
Depends: libacl1 (>= 2.2.51-8), libaudit1 (>= 1:2.2.1), libblkid1 (>=
2.19.1), libcap2 (>= 1:2.10), libcryptsetup4 (>= 2:1.4.3), libkmod2 (>=
5~), libpam0g (>= 0.99.7.1), libselinux1 (>= 2.1.9), libsystemd0 (=
215-17+deb8u2), util-linux (>= 2.19.1-2), mount (>= 2.21), initscripts
(>= 2.88dsf-53.2), sysv-rc, udev (>= 208-8), acl, adduser, libcap2-bin
Pre-Depends: libc6 (>= 2.17), libgcrypt20 (>= 1.6.1), liblzma5 (>=
5.1.1alpha+20120614), libselinux1 (>= 1.32)
Recommends: libpam-systemd, dbus
Suggests: systemd-ui
Breaks: lsb-base (<< 4.1+Debian4), lvm2 (<< 2.02.104-1), systemd-shim
(<< 8-2)
Conflicts: klogd
Conffiles:
 /etc/dbus-1/system.d/org.freedesktop.hostname1.conf
f55c94d000b5d62b5f06d38852977dd1
 /etc/dbus-1/system.d/org.freedesktop.locale1.conf
5893ab03e7e96aa3759baceb4dd04190
 /etc/dbus-1/system.d/org.freedesktop.login1.conf
96bf488f3da8f0ca813cc78e71eeb605
 /etc/dbus-1/system.d/org.freedesktop.machine1.conf
d658acaf7192de735ab798c58ab149ae
 /etc/dbus-1/system.d/org.freedesktop.systemd1.conf
46533268285a0df22b1f1667ddc05642
 /etc/dbus-1/system.d/org.freedesktop.timedate1.conf
682369fbf3de26b21e775732c89a2bbe
 /etc/pam.d/systemd-user aebb8b8a2c698614b00961f327a683ef
 /etc/systemd/bootchart.conf 838a83315ad2f9d4c4b7f20b7475ba28
 /etc/systemd/journald.conf 51f19202dcc5aff6d2c3448dbf92f354
 /etc/systemd/logind.conf 747ae2312b6922546190bf806d8d12be
 /etc/systemd/resolved.conf 81a9c57d4eb587decc49040595674c5a
 /etc/systemd/system.conf 3d0e13a72d72acb55409a831b72a15e6
 /etc/systemd/timesyncd.conf 605e4f25097b8ea22f2f5cb3fde3f13c
 /etc/systemd/user.conf 9eaad8fbb7bb6230d4b28abb76b4e81c
Description: system and service manager
 systemd is a replacement for sysvinit.  It is dependency-based and


>> I have two cases where an 'apt-get dist-upgrade' from jessie to
>> stretch calls 'systemd.postinst configure' differently:
>> * a minimal jessie chroot (incorrect behavior)
>> * a minimal jessie chroot + bash-completion (correct behavior)

and starting from that chroot.tgz I can reproduce the two upgrade paths
with the differing behavior manually:

  apt-get install bash-completion  # only for the "correct" one
  sed -i s/jessie/stretch/ /etc/apt/sources.list
  apt-get update
  apt-get dist-upgrade


If the postinst is invoked correctly the configuration of systemd ends
with the removal of an obsolete conffile:

...
Unpacking systemd (226-3) over (215-17+deb8u2) ...
Setting up util-linux (2.27-3) ...
Setting up systemd (226-3) ...
...
Removing obsolete conffile
/etc/dbus-1/system.d/org.freedesktop.machine1.conf ...
...

If the postinst is incorrectly invoked like it were an initial install,
the obsolete conffile removal is missing.


if I start with

  apt-get install bash-completion
  apt-get purge bash-completion

I also get into the incorrect behavior.



You can find the chroot at people.debian.org: ~anbe/jessie_amd64.tar.gz
You may have to nuke the proxy setting in /etc/apt/apt.conf.d/piuparts
to test this yourself.



Andreas

Bug#801156: dpkg: sometimes does not pass old version to postinst on upgrade

[Guillem Jover]

Hi!

On Wed, 2015-10-07 at 01:10:33 +0200, Andreas Beckmann wrote:
> Package: dpkg
> Version: 1.18.3
> Severity: serious

> dpkg does not always properly pass the old version number when
> calling "postinst configure" on package upgrades, sometimes there
> is either no or an empty argument.

That's to be expected, the second argument is the last version that
got configured, if no version has ever been configured then the
argument is omitted. Checking now, this is documented in Debian
policy §6.7.

> I have two cases where an 'apt-get dist-upgrade' from jessie to
> stretch calls 'systemd.postinst configure' differently:
> * a minimal jessie chroot (incorrect behavior)
> * a minimal jessie chroot + bash-completion (correct behavior)

> So in these two cases completely different codepaths in the postinst are 
> taken.
> I don't know about other packages, since this is (so far) the only one leaving
> a visible trace from this mistreatment in piuparts. (Well, there are several
> other systemd symlink related problems that nobody managed to debug so far.)

> Attached are two logs where I did this upgrade test with piuparts, but
> it should be quite easy to reproduce it manually.

I've not checked the logs, but I'd assume that in the problematic case
the package has never been configured before, and as such the bug here
is on that specific package.

Thanks,
Guillem

Bug#801156: dpkg: sometimes does not pass old version to postinst on upgrade

[Andreas Beckmann]

Package: dpkg
Version: 1.18.3
Severity: serious

Hi Guillem,

dpkg does not always properly pass the old version number when
calling "postinst configure" on package upgrades, sometimes there
is either no or an empty argument.

I have two cases where an 'apt-get dist-upgrade' from jessie to
stretch calls 'systemd.postinst configure' differently:
* a minimal jessie chroot (incorrect behavior)
* a minimal jessie chroot + bash-completion (correct behavior)

I noticed this while analyzing some strange piuparts failures in jessie
to stretch upgrades. Unfortunately I neglected it for a long time since
I assumed to have hit a strange bug in some systemd support scripts:

1m41.2s ERROR: FAIL: After purging files have disappeared:
  /etc/dbus-1/system.d/org.freedesktop.machine1.conf.dpkg-remove not 
owned
  /etc/systemd/system/halt.target.wants/ not owned
  /etc/systemd/system/halt.target.wants/hwclock-save.service -> 
/lib/systemd/system/hwclock-save.service not owned
  /etc/systemd/system/poweroff.target.wants/ not owned
  /etc/systemd/system/poweroff.target.wants/hwclock-save.service -> 
/lib/systemd/system/hwclock-save.service not owned
  /etc/systemd/system/reboot.target.wants/   not owned
  /etc/systemd/system/reboot.target.wants/hwclock-save.service -> 
/lib/systemd/system/hwclock-save.service   not owned

Digging into this I found the systemd.postinst from stretch performing
cleanup of hwclock-save.service and removal of that obsolete conffile
- but strangely only sometimes:


# Cleanup hwclock-save.service, which was shipped in jessie.
if dpkg --compare-versions "$2" lt-nl "219-8"; then
   for t in reboot halt poweroff ; do
   rm -fv /etc/systemd/system/${t}.target.wants/hwclock-save.service
   rmdir --ignore-fail-on-non-empty /etc/systemd/system/${t}.target.wants 
2> /dev/null || true
   done
fi

#DEBHELPER#

(the #DEBHELPER# part gets several dkpm-maintscript-helper rm_conffile calls)


not here, which is wrong:


(Reading database ... 7446 files and directories currently installed.)
  Preparing to unpack .../systemd_226-3_amd64.deb ...
  Unpacking systemd (226-3) over (215-17+deb8u2) ...
  Setting up util-linux (2.27-3) ...
  Setting up systemd (226-3) ...
  Installing new version of config file 
/etc/dbus-1/system.d/org.freedesktop.login1.conf ...
  Installing new version of config file 
/etc/dbus-1/system.d/org.freedesktop.systemd1.conf ...
  Installing new version of config file /etc/pam.d/systemd-user ...
  Installing new version of config file /etc/systemd/bootchart.conf ...
  Installing new version of config file /etc/systemd/journald.conf ...
  Installing new version of config file /etc/systemd/logind.conf ...
  Installing new version of config file /etc/systemd/resolved.conf ...
  Installing new version of config file /etc/systemd/system.conf ...
  Installing new version of config file /etc/systemd/timesyncd.conf ...
  Installing new version of config file /etc/systemd/user.conf ...
  Created symlink from 
/etc/systemd/system/sysinit.target.wants/systemd-timesyncd.service to 
/lib/systemd/system/systemd-timesyncd.service.
  addgroup: The group `systemd-journal' already exists as a system group. 
Exiting.
  (Reading database ... 


but here, which is correct:


(Reading database ... 8034 files and directories currently installed.)
  Preparing to unpack .../systemd_226-3_amd64.deb ...
  Unpacking systemd (226-3) over (215-17+deb8u2) ...
  Setting up util-linux (2.27-3) ...
  Setting up systemd (226-3) ...
  Installing new version of config file 
/etc/dbus-1/system.d/org.freedesktop.login1.conf ...
  Installing new version of config file 
/etc/dbus-1/system.d/org.freedesktop.systemd1.conf ...
  Installing new version of config file /etc/pam.d/systemd-user ...
  Installing new version of config file /etc/systemd/bootchart.conf ...
  Installing new version of config file /etc/systemd/journald.conf ...
  Installing new version of config file /etc/systemd/logind.conf ...
  Installing new version of config file /etc/systemd/resolved.conf ...
  Installing new version of config file /etc/systemd/system.conf ...
  Installing new version of config file /etc/systemd/timesyncd.conf ...
  Installing new version of config file /etc/systemd/user.conf ...
  Created symlink from 
/etc/systemd/system/sysinit.target.wants/systemd-timesyncd.service to 
/lib/systemd/system/systemd-timesyncd.service.
  addgroup: The group `systemd-journal' already exists as a system group. 
Exiting.
  Removing obsolete conffile /etc/dbus-1/system.d/org.freedesktop.machine1.conf 
...
  (Reading database ... 



So I rebuilt systemd with more debug output in the postinst script to see the 
following:


  (Reading database ... 7446 files and directories currently installed.)
  Preparing to unpack ..././systemd_226-4.1_amd64.deb ...
  Unpacking systemd (226-4.1) over (215-17+deb8u2) ...
  Setting up util-linux (2.27-3) ...
  Setting up systemd (226-4.1) ...
  Instal