Re: service failures should not fail dpkg installation [was: Re: promoting virtualbox-dkms to virtualbox pre-depends]

[Jeroen Dekkers]

At Thu, 24 Sep 2015 08:11:48 -0400,
Marvin Renich wrote:
> 
> * Jeroen Dekkers  [150924 07:23]:
> > At Wed, 23 Sep 2015 13:53:11 -0400,
> > Marvin Renich wrote:
> > > I think it should be documented in the developers reference that if you
> > > attempt to start or restart a service in postinst, you should guard it
> > > so that a failure in the service does not propagate to a failure of the
> > > postinst.
> > 
> > But then when something goes wrong when upgrading and the service
> > doesn't (re)start apt/dpkg will report success but the service isn't
> > running anymore. That also sounds wrong to me. Letting postinst fail
> > might not be the best way to signal this, but to change that we need
> > something else to let the user know that something went wrong. Just
> > printing an error message isn't enough, because the user might not see
> > that (for example when multiple packages are installed/upgraded and a
> > later package asks some questions using dialog or when using
> > unattended-upgrades).
> 
> How does failing the upgrade solve anything?  The upgrade should only
> fail if the failure of the service to start was because something in the
> upgrade itself was broken; this is rarely the case.

I think it solves the problem of notifying the user that something
went wrong quite clearly. Not in the correct way, I agree with that,
but the solution to that should be to notify the user in a better way,
not to stop notifying the user. Failing silently is worse than failing
in the wrong way.

> What makes this even worse is that when installing or upgrading a large
> number of packages, this kind of incorrect failure sometimes affects
> many completely unrelated packages.  For an unattended upgrade, this is
> so much worse than having one service that (for a correct reason)
> refused to restart after the upgrade.

Unattended-upgrades has the MinimalSteps option that splits upgrades
in the smallest possible chunks so that isn't really a problem.
 
> What you are looking for is a more prominent notification that a service
> did not restart.  But the current situation is like the "check engine"
> light flashing when you are low on fuel; yes, it gets your attention,
> but it is telling you the wrong thing.

Yes, but the way to solve that is to flash a "low on fuel" light, not
to stop notifying you and leaving you alone in the desert without
fuel. And if a "low on fuel" light isn't possible, it's better to keep
flashing the "check engine" light like it has been doing for the past
15 years.


Kind regards,

Jeroen Dekkers

Re: promoting virtualbox-dkms to virtualbox pre-depends

[Jeroen Dekkers]

At Wed, 23 Sep 2015 13:53:11 -0400,
Marvin Renich wrote:
> I think it should be documented in the developers reference that if you
> attempt to start or restart a service in postinst, you should guard it
> so that a failure in the service does not propagate to a failure of the
> postinst.

But then when something goes wrong when upgrading and the service
doesn't (re)start apt/dpkg will report success but the service isn't
running anymore. That also sounds wrong to me. Letting postinst fail
might not be the best way to signal this, but to change that we need
something else to let the user know that something went wrong. Just
printing an error message isn't enough, because the user might not see
that (for example when multiple packages are installed/upgraded and a
later package asks some questions using dialog or when using
unattended-upgrades).

Kind regards,

Jeroen Dekkers

Re: promoting virtualbox-dkms to virtualbox pre-depends

[Thomas Goirand]

On 09/22/2015 06:40 PM, Gianfranco Costamagna wrote:
> Pre-Depends: dpkg (>= 1.15.6~)

This pre-depend is for ancient version of dpkg (ie: only useful for
Precise which did not support xz compression). I'd advise to drop it at
this point (I've been dropping it on all of my packages on each upload).

Cheers,

Thomas Goirand (zigo)

service failures should not fail dpkg installation [was: Re: promoting virtualbox-dkms to virtualbox pre-depends]

[Marvin Renich]

* Jeroen Dekkers  [150924 07:23]:
> At Wed, 23 Sep 2015 13:53:11 -0400,
> Marvin Renich wrote:
> > I think it should be documented in the developers reference that if you
> > attempt to start or restart a service in postinst, you should guard it
> > so that a failure in the service does not propagate to a failure of the
> > postinst.
> 
> But then when something goes wrong when upgrading and the service
> doesn't (re)start apt/dpkg will report success but the service isn't
> running anymore. That also sounds wrong to me. Letting postinst fail
> might not be the best way to signal this, but to change that we need
> something else to let the user know that something went wrong. Just
> printing an error message isn't enough, because the user might not see
> that (for example when multiple packages are installed/upgraded and a
> later package asks some questions using dialog or when using
> unattended-upgrades).

How does failing the upgrade solve anything?  The upgrade should only
fail if the failure of the service to start was because something in the
upgrade itself was broken; this is rarely the case.

There are two prominent reasons why a service fails to start after an
upgrade:  the relationship between the application and its configuration
has changed (e.g. different, incompatible defaults or incompatible file
format) or some external influence that has nothing to do with the
upgrade (e.g.  unavailable resource).

The first case requires the admin to sort out the changes and fix the
configuration.  Being required to re-run the dpkg installation just to
flip the 'half-configured' state to 'installed' when the result would
have been the same if dpkg had not failed the first time is wrong.

In the second case, how is it a dpkg installation failure if the
hypervisor is not running so xen won't start?  Everything is installed
perfectly.  Or if a daemon fails to start because the ldap server on a
different host is down?  Failing the installation is _really_, _really_
_wrong_!

What makes this even worse is that when installing or upgrading a large
number of packages, this kind of incorrect failure sometimes affects
many completely unrelated packages.  For an unattended upgrade, this is
so much worse than having one service that (for a correct reason)
refused to restart after the upgrade.

What you are looking for is a more prominent notification that a service
did not restart.  But the current situation is like the "check engine"
light flashing when you are low on fuel; yes, it gets your attention,
but it is telling you the wrong thing.

...Marvin

Re: service failures should not fail dpkg installation [was: Re: promoting virtualbox-dkms to virtualbox pre-depends]

[Henrique de Moraes Holschuh]

On Thu, 24 Sep 2015, Marvin Renich wrote:
> How does failing the upgrade solve anything?  The upgrade should only
> fail if the failure of the service to start was because something in the
> upgrade itself was broken; this is rarely the case.

...

> What makes this even worse is that when installing or upgrading a large
> number of packages, this kind of incorrect failure sometimes affects
> many completely unrelated packages.  For an unattended upgrade, this is
> so much worse than having one service that (for a correct reason)
> refused to restart after the upgrade.

What we really want is a "do not fail upgrade, BUT report that some services
*that were previously running* failed to restart after the upgrade run".

ESPECIALLY if you are going to take "unattended upgrades" seriously.

Still, that would need some proper design work, and a reasonable amount of
code to be written and tested.  Some of it will hook into the package
system, some of it needs to interface to the services subsystem (systemd,
sysvinit, others).

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh

Re: service failures should not fail dpkg installation [was: Re: promoting virtualbox-dkms to virtualbox pre-depends]

[Paul Gevers]

Hi

On 24-09-15 18:21, Henrique de Moraes Holschuh wrote:
> On Thu, 24 Sep 2015, Marvin Renich wrote:
> What we really want is a "do not fail upgrade, BUT report that some services
> *that were previously running* failed to restart after the upgrade run".
> 
> ESPECIALLY if you are going to take "unattended upgrades" seriously.
> 
> Still, that would need some proper design work, and a reasonable amount of
> code to be written and tested.  Some of it will hook into the package
> system, some of it needs to interface to the services subsystem (systemd,
> sysvinit, others).

I would like to add there is more than just services. As the current
maintainer of dbconfig-common, it is more than clear to me that updates
of packages that require updates of (and even installs into) databases
(tables and/or their contents) also fall into this category. If for
whatever reason we can't connect to the database (which may even be on a
different system), there is currently not much that we can do except
register failure. I am currently of the opinion that if that happens,
the package upgrade DID fail, as the package probably won't be working
until the upgrade commands are applied with a working connection. (Just
before people start shouting, the way dbconfig-common handles this is by
asking the administrator if the problem should be fixed by retrying,
ignoring the problem or considering the issue a failure. In
noninteractive mode, the problem is ignored for installs and removals,
but not for upgrades.)

Paul



signature.asc
Description: OpenPGP digital signature

Re: promoting virtualbox-dkms to virtualbox pre-depends

[Ritesh Raj Sarraf]

On Wed, 2015-09-23 at 13:23 +0200, Ansgar Burchardt wrote:
> Gianfranco Costamagna  writes:
> > the problem actually is that virtualbox-dkms should be configured
> > *before* configuring virtualbox
> 
> Why should this need Pre-Depends instead of Depends assuming
> virtualbox
> depends on the -dkms package?

Hello Ansgar,

Please see attached terminal log from APT. I have copied the relevant
sections. It shows the purpose of vbox-dkms and why exactly virtualbox
package fails.

Also, the same has been discussed in following bug reports.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=798979
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=798527


-- 
Ritesh Raj Sarraf | http://people.debian.org/~rrs
Debian - The Universal Operating System
Log started: 2015-09-15  14:09:20
Selecting previously unselected package aptitude-common.
(Reading database ... 
(Reading database ... 5%
(Reading database ... 10%
(Reading database ... 15%
(Reading database ... 20%
(Reading database ... 25%
(Reading database ... 30%
(Reading database ... 35%
(Reading database ... 40%
(Reading database ... 45%
(Reading database ... 50%
(Reading database ... 55%
(Reading database ... 60%
(Reading database ... 65%
(Reading database ... 70%
(Reading database ... 75%
(Reading database ... 80%
(Reading database ... 85%
(Reading database ... 90%
(Reading database ... 95%
(Reading database ... 100%
(Reading database ... 439671 files and directories currently installed.)
Preparing to unpack .../aptitude-common_0.7.1-1_all.deb ...
Unpacking aptitude-common (0.7.1-1) ...
Selecting previously unselected package libcwidget3v5:amd64.
Preparing to unpack .../libcwidget3v5_0.5.17-4_amd64.deb ...
Unpacking libcwidget3v5:amd64 (0.5.17-4) ...
Selecting previously unselected package aptitude.
Preparing to unpack .../aptitude_0.7.1-1_amd64.deb ...
Unpacking aptitude (0.7.1-1) ...
Processing triggers for man-db (2.7.3-1) ...
Processing triggers for menu (2.1.47) ...
Setting up aptitude-common (0.7.1-1) ...
Setting up libcwidget3v5:amd64 (0.5.17-4) ...
Setting up aptitude (0.7.1-1) ...
update-alternatives: using /usr/bin/aptitude-curses to provide /usr/bin/aptitude (aptitude) in auto mode
Processing triggers for libc-bin (2.19-19) ...
Processing triggers for menu (2.1.47) ...
Log ended: 2015-09-15  14:09:26

Log started: 2015-09-15  19:07:46
Selecting previously unselected package python-gpgme.
(Reading database ... 
(Reading database ... 5%
(Reading database ... 10%
(Reading database ... 15%
(Reading database ... 20%
(Reading database ... 25%
(Reading database ... 30%
(Reading database ... 35%
(Reading database ... 40%
(Reading database ... 45%
(Reading database ... 50%
(Reading database ... 55%
(Reading database ... 60%
(Reading database ... 65%
(Reading database ... 70%
(Reading database ... 75%
(Reading database ... 80%
(Reading database ... 85%
(Reading database ... 90%
(Reading database ... 95%
(Reading database ... 100%
(Reading database ... 439876 files and directories currently installed.)
Preparing to unpack .../python-gpgme_0.3-1+b1_amd64.deb ...
Unpacking python-gpgme (0.3-1+b1) ...
Selecting previously unselected package bmap-tools.
Preparing to unpack .../bmap-tools_3.2-2_all.deb ...
Unpacking bmap-tools (3.2-2) ...
Processing triggers for man-db (2.7.3-1) ...
Setting up python-gpgme (0.3-1+b1) ...
Setting up bmap-tools (3.2-2) ...
Log ended: 2015-09-15  19:07:48

Log started: 2015-09-16  11:53:57
(Reading database ... 
(Reading database ... 5%
(Reading database ... 10%
(Reading database ... 15%
(Reading database ... 20%
(Reading database ... 25%
(Reading database ... 30%
(Reading database ... 35%
(Reading database ... 40%
(Reading database ... 45%
(Reading database ... 50%
(Reading database ... 55%
(Reading database ... 60%
(Reading database ... 65%
(Reading database ... 70%
(Reading database ... 75%
(Reading database ... 80%
(Reading database ... 85%
(Reading database ... 90%
(Reading database ... 95%
(Reading database ... 100%
(Reading database ... 439904 files and directories currently installed.)
Preparing to unpack .../virtualbox-qt_5.0.4-dfsg-2_amd64.deb ...
Unpacking virtualbox-qt (5.0.4-dfsg-2) over (5.0.2-dfsg-2) ...
Preparing to unpack .../virtualbox_5.0.4-dfsg-2_amd64.deb ...
Unpacking virtualbox (5.0.4-dfsg-2) over (5.0.2-dfsg-2) ...
Preparing to unpack .../virtualbox-dkms_5.0.4-dfsg-2_all.deb ...

 Uninstall Beginning 
Module:  virtualbox
Version: 5.0.2
Kernel:  4.1.6+ (x86_64)
-

Status: Before uninstall, this module version was ACTIVE on this kernel.

vboxdrv.ko:
 - Uninstallation
   - Deleting from: /lib/modules/4.1.6+/updates/dkms/
 - Original module
   - No original module was found for this module on this kernel.
   - Use the dkms install command to reinstall any previous module version.


vboxnetadp.ko:
 - Uninstallation
   - Deleting from: /lib/modules/4.1.6+/updates/dkms/
 - Original module
   - No original module was found for this 

Re: promoting virtualbox-dkms to virtualbox pre-depends

[Marc Haber]

On Wed, 23 Sep 2015 12:31:22 +0200, Gianfranco Costamagna
 wrote:
>well, in that case you can download the source package, build the dkms
>module and then install virtualbox, right?
>
>if you run a custom kernel you should know how to build a dkms module.

Do we now require DKMS, which requires a fully-blown build system on
the target box? Wouldn't it be much better to be able to build a
virtualbox-modules-$KERNELFLAVOUR.deb? What was so bad with
module-assistant?

Greetings
Marc
-- 
-- !! No courtesy copies, please !! -
Marc Haber |   " Questions are the | Mailadresse im Header
Mannheim, Germany  | Beginning of Wisdom " | http://www.zugschlus.de/
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 72739834

Re: promoting virtualbox-dkms to virtualbox pre-depends

[Ansgar Burchardt]

Gianfranco Costamagna  writes:
> the problem actually is that virtualbox-dkms should be configured
> *before* configuring virtualbox

Why should this need Pre-Depends instead of Depends assuming virtualbox
depends on the -dkms package?

Ansgar

Re: promoting virtualbox-dkms to virtualbox pre-depends

[Marvin Renich]

* Ritesh Raj Sarraf  [150923 07:47]:
> On Wed, 2015-09-23 at 13:23 +0200, Ansgar Burchardt wrote:
> > Gianfranco Costamagna  writes:
> > > the problem actually is that virtualbox-dkms should be configured
> > > *before* configuring virtualbox
> > 
> > Why should this need Pre-Depends instead of Depends assuming
> > virtualbox
> > depends on the -dkms package?
> 
> Hello Ansgar,
> 
> Please see attached terminal log from APT. I have copied the relevant
> sections. It shows the purpose of vbox-dkms and why exactly virtualbox
> package fails.
> 
> Also, the same has been discussed in following bug reports.
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=798979
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=798527

The logs (that I snipped) were from an installation of the versions
where virtualbox-dkms incorrectly had a Depends on virtualbox.  This
does nothing to justify using Pre-Depends.  Using Pre-Depends is wrong.

...Marvin

Re: Re: promoting virtualbox-dkms to virtualbox pre-depends

[Gianfranco Costamagna]

well, in that case you can download the source package, build the dkms
module and then install virtualbox, right?

if you run a custom kernel you should know how to build a dkms module.

cheers,

G.

Re: promoting virtualbox-dkms to virtualbox pre-depends

[Marvin Renich]

* Ritesh Raj Sarraf  [150923 04:06]:
> Adding Moritz from the DSA.
> 
> On Tue, 2015-09-22 at 19:59 +0200, Julien Cristau wrote:
> > On Tue, Sep 22, 2015 at 15:00:35 +, Gianfranco Costamagna wrote:
> > 
> > > As shown in policy 7.2
> > > 
> > > "You should not specify a Pre-Depends entry for a package before
> > this has been discussed on the debian-devel mailing
> > > list and a consensus about doing that has been reached. See
> > Dependencies, Section 3.5."
> > > 
> > > the problem actually is that virtualbox-dkms should be configured
> > *before* configuring virtualbox, otherwise
> > > without a built kernel module, restarting VMs
> > > will fail and lead to an half-configured package.
> > > 
> > > Please see bugs #798527 and #798979 as examples.
> > > 
> > > (this is a subpackage depending on another subpackage that belongs
> > to the same src, I don't get why I should discuss such internal
> > > changes, but well, policy is policy)
> > > 
> > A pre-depends is very much the wrong hammer here, userspace can't
> > usefully rely on a kernel package or module through package
> > dependencies.
> 
> Can you please elaborate here ?
> 
> 
> Problem is: virtualbox is picky about the version of the kernel module
> in use. Which is provided by virtualbox-dkms package.
> 
> We earlier did have the Depends logic, which broke. I don't think the
> breakage was racy. It was something just not noticed commonly.
> 
> By putting the tighter dependency, we are instructing virtualbox wait
> until virtualbox-dkms has completed its installation, which effectively
> is: roll out the new kernel dkms package, built the new kernel modules,
> and load them. That is exactly what virtualbox package will need before
> it can upgrade itself.
> 
> 
> Looking at the Pre-Depends details on the policy document, it looks in
> line with what we want:
> 
[policy quote snipped]

No.  Pre-Depends says to unpack and configure another package before
even unpacking this package.  Depends says to make sure the other
package is both unpacked and configured before _configuring_ this
package, but it is okay to unpack this package prior to that.

Pre-Depends is truly the wrong hammer here, regardless of whether we are
talking about kernel modules or normal user software.

The problem originally was that you had the Depends the wrong way
around.  You had the vbox-dkms package with a Depends on the vbox
package (which was completely wrong; there was no need for that at all),
and the vbox package with a Recommends on vbox-dkms.  This forced vbox
to be both unpacked and configured before vbox-dkms was configured,
which caused your problem.

Another poster's comment was that just raising the vbox Recommends
vbox-dkms to a Depends would create a circular dependency, which would
not guarantee that vbox-dkms was configured first.  However, both
raising vbox to Depend (_not_ Pre-Depend) on vbox-dkms _and_ lowering
the vbox-dkms Depends vbox to a Recommends breaks the dependency cycle
and forces vbox-dkms to be configured first, which is what you are
trying to accomplish.

On the other hand, other posters here have said that package-level
dependencies are the wrong hammer for enforcing kernel module
dependencies, and they are right.

What might work, as a better kludge than using Depends to mean "in most
cases when vbox is installed, vbox-dkms or vbox-source should also be
installed [which is the definition of Recommends], and if either is
installed, we want the module to be built and loaded before starting the
vbox service [which is what you are trying to accomplish using
Pre-Depends]", is to use Recommends in both places, which expresses the
correct package relationship, and then use dpkg triggers to start the
vbox service after all packages in the dpkg installation run have been
configured.

This works whether the user has a custom kernel or a stock Debian
kernel, as long as they have the module installed one way or another.
And if they don't, they will get the appropriate error message from the
vbox service.

I have not used dpkg triggers myself, so I may have this wrong, but this
is what I would try.  There may be a better solution, but I can't think
of one at the moment.

Pre-Depends is highly misunderstood, and this thread is a good example
of why policy dictates that using it should be discussed on this list
first.

...Marvin

Re: promoting virtualbox-dkms to virtualbox pre-depends

[Marvin Renich]

* Marvin Renich  [150923 07:52]:
> What might work, ..., and then use dpkg triggers to start the
> vbox service after all packages in the dpkg installation run have been
> configured.

If I understand correctly how dpkg triggers work, this could cause a
restart of vbox before the module is installed, but then it should also
cause a second restart after it is installed.  (I.e. you could get a
spurious failing restart in the middle, but when the dust settles, all
should be well.)

This is caused by apt and aptitude breaking up large installations into
multiple calls to dpkg.  Each call to dpkg will cause the triggers to be
run, so if apt(itude) happens to separate the vbox and vbox-dkms
installations into separate runs of dpkg, vbox might be restarted the
first time with the wrong module, but it will be corrected in a
subsequent run.  This assumes correctly using Recommends instead of
Depends.  As I (and others) said earlier, not only is Pre-Depends wrong,
but Depends is also wrong in this situation.

...Marvin

Re: promoting virtualbox-dkms to virtualbox pre-depends

[Ritesh Raj Sarraf]

On Wed, 2015-09-23 at 08:18 -0400, Marvin Renich wrote:
> If I understand correctly how dpkg triggers work, this could cause a
> restart of vbox before the module is installed, but then it should
> also
> cause a second restart after it is installed.  (I.e. you could get a
> spurious failing restart in the middle, but when the dust settles,
> all
> should be well.)
> 
> This is caused by apt and aptitude breaking up large installations
> into
> multiple calls to dpkg.  Each call to dpkg will cause the triggers to
> be
> run, so if apt(itude) happens to separate the vbox and vbox-dkms
> installations into separate runs of dpkg, vbox might be restarted the
> first time with the wrong module, but it will be corrected in a
> subsequent run.  This assumes correctly using Recommends instead of
> Depends.  As I (and others) said earlier, not only is Pre-Depends
> wrong,
> but Depends is also wrong in this situation.

Hmmm.. I think I have seen this kind of error some time ago.

So how is such error supposed to be treated ? Ignore it ? I guess the
final result of apt is still a success, in such cases.


-- 
Given the large number of mailing lists I follow, I request you to CC
me in replies for quicker response



signature.asc
Description: This is a digitally signed message part

Re: promoting virtualbox-dkms to virtualbox pre-depends

[Ritesh Raj Sarraf]

On Wed, 2015-09-23 at 15:02 +0200, Marc Haber wrote:
> Do we now require DKMS, which requires a fully-blown build system on
> the target box? Wouldn't it be much better to be able to build a
> virtualbox-modules-$KERNELFLAVOUR.deb? What was so bad with
> module-assistant?

I think it is still there. But given the features of dkms, nobody must
be missing it.


-- 
Given the large number of mailing lists I follow, I request you to CC
me in replies for quicker response



signature.asc
Description: This is a digitally signed message part

Re: promoting virtualbox-dkms to virtualbox pre-depends

[Ritesh Raj Sarraf]

On Wed, 2015-09-23 at 08:18 -0400, Marvin Renich wrote:
> * Marvin Renich  [150923 07:52]:
> > What might work, ..., and then use dpkg triggers to start the
> > vbox service after all packages in the dpkg installation run have
> been
> > configured.
> 
> If I understand correctly how dpkg triggers work, this could cause a
> restart of vbox before the module is installed, but then it should
> also
> cause a second restart after it is installed.  (I.e. you could get a
> spurious failing restart in the middle, but when the dust settles,
> all
> should be well.)
> 
> This is caused by apt and aptitude breaking up large installations
> into
> multiple calls to dpkg.  Each call to dpkg will cause the triggers to
> be
> run, so if apt(itude) happens to separate the vbox and vbox-dkms
> installations into separate runs of dpkg, vbox might be restarted the
> first time with the wrong module, but it will be corrected in a
> subsequent run.  This assumes correctly using Recommends instead of
> Depends.  As I (and others) said earlier, not only is Pre-Depends
> wrong,
> but Depends is also wrong in this situation.

Thanks for the explanation Marvin. We'll test and see what fits best.

-- 
Given the large number of mailing lists I follow, I request you to CC
me in replies for quicker response



signature.asc
Description: This is a digitally signed message part

Re: Re: promoting virtualbox-dkms to virtualbox pre-depends

[Gianfranco Costamagna]

Hi, confirmed: it is still there.

cheers,

Gianfranco

Re: promoting virtualbox-dkms to virtualbox pre-depends

[Ritesh Raj Sarraf]

On Wed, 2015-09-23 at 08:07 -0400, Marvin Renich wrote:
> > Also, the same has been discussed in following bug reports.
> > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=798979
> > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=798527
> 
> The logs (that I snipped) were from an installation of the versions
> where virtualbox-dkms incorrectly had a Depends on virtualbox.  This
> does nothing to justify using Pre-Depends.  Using Pre-Depends is
> wrong.

Okay!!  But, btw, can someone enlighten the use case for Pre-Depends ??

-- 
Given the large number of mailing lists I follow, I request you to CC
me in replies for quicker response



signature.asc
Description: This is a digitally signed message part

Re: promoting virtualbox-dkms to virtualbox pre-depends

[Marvin Renich]

[please do not CC me; I am subscribed]

* Ritesh Raj Sarraf  [150923 11:50]:
> Okay!!  But, btw, can someone enlighten the use case for Pre-Depends ??

If you need to use something from another package in your preinst
script, you will need a Pre-Depends.  For example, cron Pre-Depends:
dpkg because the preinst script for cron uses dpkg-maintscript-helper
which is in dpkg.

...Marvin

Re: promoting virtualbox-dkms to virtualbox pre-depends

[Marc Haber]

On Wed, 23 Sep 2015 21:14:13 +0530, Ritesh Raj Sarraf 
wrote:
>On Wed, 2015-09-23 at 15:02 +0200, Marc Haber wrote:
>> Do we now require DKMS, which requires a fully-blown build system on
>> the target box? Wouldn't it be much better to be able to build a
>> virtualbox-modules-$KERNELFLAVOUR.deb? What was so bad with
>> module-assistant?
>
>I think it is still there. But given the features of dkms, nobody must
>be missing it.

Well, I am. Having a fully functional build system on a productive
server is a total no-go.

Greetings
Marc
-- 
-- !! No courtesy copies, please !! -
Marc Haber |   " Questions are the | Mailadresse im Header
Mannheim, Germany  | Beginning of Wisdom " | http://www.zugschlus.de/
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 72739834

Re: promoting virtualbox-dkms to virtualbox pre-depends

[Marvin Renich]

* Ritesh Raj Sarraf  [150923 12:54]:
> > Each call to dpkg will cause the triggers to be run, so if
> > apt(itude) happens to separate the vbox and vbox-dkms installations
> > into separate runs of dpkg, vbox might be restarted the first time
> > with the wrong module, but it will be corrected in a subsequent run.
> 
> Hmmm.. I think I have seen this kind of error some time ago.
> 
> So how is such error supposed to be treated ? Ignore it ? I guess the
> final result of apt is still a success, in such cases.

First, you should start your service in the trigger, not in your
postinst.  From the dpkg man page (I am really stretching with this
inference, so someone else can clarify) it looks like a failure in the
trigger will leave the package in the triggers-pending state.  However,
you should make sure the trigger does not fail even if the vbox service
does not start correctly.



>From the first time I had dpkg mark a package as half-configured when
everything was correct except that the service would not start for some
reason that had nothing to do with package installation (exactly the
situation here for virtualbox), I have felt that dpkg had no business
failing just because the service would not start.  I think that is a
wrong design decision.

In fact, one specific case that often hurts me is when I have xen
installed on a machine where I only run the hypervisor occasionally.
Upgrading the xen packages causes (or has caused in the past) the
upgrade to fail.  This is ridiculous!

I think it should be documented in the developers reference that if you
attempt to start or restart a service in postinst, you should guard it
so that a failure in the service does not propagate to a failure of the
postinst.



...Marvin

Re: promoting virtualbox-dkms to virtualbox pre-depends

[Adam Borowski]

On Wed, Sep 23, 2015 at 08:17:44PM +0200, Eduard Bloch wrote:
> And DKMS... oh, dear. Did they eventually manage to fix it, to not
> blindly rely on /lib/modules/.../source (or build) symlinks as the
> one and only source of information? Not having those links is like a
> normal case with kernel+header packages built with make-kpkg on a fast
> remote remote machine.

The interaction of kernel-package with dkms is currently broken the opposite
way: building on a remote machine works, building on the one you install
kernel-headers on is broken: it will notice the path you built the kernel in
exists and instead of /usr/src/linux-headers-$VERSION it will make links to
the build path.  This will work only as long that path contains the same
kernel version.

-- 
⢎⣉⠂⠠⠤⡀⣄⠤⡀⠠⡅⠀⠤⡧⠄⡄⠀⡄⠀⠀⠀⠠⡅⠀⡠⠤⠄⠀⠀⠀⢴⠍⠀⡠⠤⡀⣄⠤⡀⠀⠀⠀⠤⡧⠄⣇⠤⡀⡠⠤⡀⠀⠀⠀⡄⠀⡄⡠⠤⡀⠠⠤⡀⡇⡠⠄⠀⠀⠀
⠢⠤⠃⠪⠭⠇⠇⠀⠇⠀⠣⠀⠀⠣⠄⠨⠭⠃⠣⠀⠬⠭⠂⠀⠀⠀⠸⠀⠀⠣⠤⠃⠇⠀⠀⠣⠄⠇⠀⠇⠫⠭⠁⠀⠀⠀⠣⠣⠃⠫⠭⠁⠪⠭⠇⠏⠢⠄⠀⠄⠀
(https://github.com/kilobyte/braillefont for this hack)

Re: promoting virtualbox-dkms to virtualbox pre-depends

[Marc Haber]

[module-assistant]

On Wed, 23 Sep 2015 18:24:54 +0200, Gianfranco Costamagna
 wrote:
>Hi, confirmed: it is still there.

The Question is: Is it still supported?

Greetings
Marc
-- 
-- !! No courtesy copies, please !! -
Marc Haber |   " Questions are the | Mailadresse im Header
Mannheim, Germany  | Beginning of Wisdom " | http://www.zugschlus.de/
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 72739834

Re: promoting virtualbox-dkms to virtualbox pre-depends

[Andrey Rahmatullin]

On Wed, Sep 23, 2015 at 01:35:21PM +0530, Ritesh Raj Sarraf wrote:
> > A pre-depends is very much the wrong hammer here, userspace can't
> > usefully rely on a kernel package or module through package
> > dependencies.
> 
> Can you please elaborate here ?
> 
> 
> Problem is: virtualbox is picky about the version of the kernel module
> in use. Which is provided by virtualbox-dkms package.
In a few words: you cannot be sure that the user's kernel is managed by
Debian packages, including the DKMS system, so the assumption above is not
true. 


-- 
WBR, wRAR

Re: promoting virtualbox-dkms to virtualbox pre-depends

[Ritesh Raj Sarraf]

Adding Moritz from the DSA.

On Tue, 2015-09-22 at 19:59 +0200, Julien Cristau wrote:
> On Tue, Sep 22, 2015 at 15:00:35 +, Gianfranco Costamagna wrote:
> 
> > As shown in policy 7.2
> > 
> > "You should not specify a Pre-Depends entry for a package before
> this has been discussed on the debian-devel mailing
> > list and a consensus about doing that has been reached. See
> Dependencies, Section 3.5."
> > 
> > the problem actually is that virtualbox-dkms should be configured
> *before* configuring virtualbox, otherwise
> > without a built kernel module, restarting VMs
> > will fail and lead to an half-configured package.
> > 
> > Please see bugs #798527 and #798979 as examples.
> > 
> > (this is a subpackage depending on another subpackage that belongs
> to the same src, I don't get why I should discuss such internal
> > changes, but well, policy is policy)
> > 
> A pre-depends is very much the wrong hammer here, userspace can't
> usefully rely on a kernel package or module through package
> dependencies.

Can you please elaborate here ?


Problem is: virtualbox is picky about the version of the kernel module
in use. Which is provided by virtualbox-dkms package.

We earlier did have the Depends logic, which broke. I don't think the
breakage was racy. It was something just not noticed commonly.

By putting the tighter dependency, we are instructing virtualbox wait
until virtualbox-dkms has completed its installation, which effectively
is: roll out the new kernel dkms package, built the new kernel modules,
and load them. That is exactly what virtualbox package will need before
it can upgrade itself.


Looking at the Pre-Depends details on the policy document, it looks in
line with what we want:

Pre-DependsThis field is like Depends, except that it also forces dpkg
 to complete installation of the packages named before even starting
the installation of the package which declares the pre-dependency, as
follows:When a package declaring a pre-dependency is about to be
 unpacked the pre-dependency can be satisfied if the depended-on
package is either fully configured, or even if the depended-on
package(s) are only in the "Unpacked" or the "Half-Configured" state,
provided that they have been configured correctly at some point in the
past (and not removed or partially removed since). In this case, both
the previously-configured and currently "Unpacked" or "Half-Configured"
versions must satisfy any version clause in the Pre-Depends field.When
the package declaring a pre-dependency is about to be configured, the
pre-dependency will be treated as a normal Depends. It will be
considered satisfied only if the depended-on package has been correctly
configured. However, unlike with Depends,Pre-Depends does not permit
circular dependencies to be broken. If a circular dependency is
encountered while attempting to honor Pre-Depends, the installation
will be aborted.Pre-Depends are also required if the preinst script
depends on the named package. It is best to avoid this situation if
possible.Pre-Depends should be used sparingly, preferably only by
packages whose premature upgrade or installation would hamper the
ability of the system to continue with any upgrade that might be in
progress.You should not specify a Pre-Depends entry for a package
before this has been discussed on the debian-devel mailing list and a
consensus about doing that has been reached. See Dependencies, Section
3.5.


-- 
Ritesh Raj Sarraf | http://people.debian.org/~rrs
Debian - The Universal Operating System


signature.asc
Description: This is a digitally signed message part

Re: promoting virtualbox-dkms to virtualbox pre-depends

[Eduard Bloch]

Hallo,
* Marc Haber [Wed, Sep 23 2015, 07:55:58PM]:
> [module-assistant]
> 
> On Wed, 23 Sep 2015 18:24:54 +0200, Gianfranco Costamagna
>  wrote:
> >Hi, confirmed: it is still there.
> 
> The Question is: Is it still supported?

I just typed "m-a a-i virtualbox" and it got me all modules installed
for my custom kernel. Admittedly, I haven't invested enough time into
its further development in the last years but it should be robust enough
to survive for a while.

And DKMS... oh, dear. Did they eventually manage to fix it, to not
blindly rely on /lib/modules/.../source (or build) symlinks as the
one and only source of information? Not having those links is like a
normal case with kernel+header packages built with make-kpkg on a fast
remote remote machine.

Regards,
Eduard.
-- 
 Und im irc hat man dir noch nie eine manpage lesen ans Herz gelegt?
 habe nie gerafft was die wollen

Re: promoting virtualbox-dkms to virtualbox pre-depends

[Sune Vuorela]

On 2015-09-22, Gianfranco Costamagna  wrote:
> -Pre-Depends: dpkg (>= 1.15.6~)
> +Pre-Depends: dpkg (>= 1.15.6~), virtualbox-dkms (= ${source:Version}) | 
> virtualbox-source (= ${source:Version})

I don't see what it would fix to have virtualbox-source installed at any
point help in anything.

And note that one can run kernels that doesn't do dkms.

/Sune

Re: promoting virtualbox-dkms to virtualbox pre-depends

[Julien Cristau]

On Tue, Sep 22, 2015 at 15:00:35 +, Gianfranco Costamagna wrote:

> As shown in policy 7.2
> 
> "You should not specify a Pre-Depends entry for a package before this has 
> been discussed on the debian-devel mailing
> list and a consensus about doing that has been reached. See Dependencies, 
> Section 3.5."
> 
> the problem actually is that virtualbox-dkms should be configured *before* 
> configuring virtualbox, otherwise
> without a built kernel module, restarting VMs
> will fail and lead to an half-configured package.
> 
> Please see bugs #798527 and #798979 as examples.
> 
> (this is a subpackage depending on another subpackage that belongs to the 
> same src, I don't get why I should discuss such internal
> changes, but well, policy is policy)
> 
A pre-depends is very much the wrong hammer here, userspace can't
usefully rely on a kernel package or module through package
dependencies.

Cheers,
Julien


signature.asc
Description: Digital signature

Re: promoting virtualbox-dkms to virtualbox pre-depends

[James Cowgill]

Hi,

On Tue, 2015-09-22 at 16:40 +, Gianfranco Costamagna wrote:
> Hi  James!
> 
> > This should work with a normal Depends relation (reread Policy 7.2).
> > 
> > This commit helped since it prevented a circular dependency involving
> > the two packages:
> > https://anonscm.debian.org/cgit/pkg-virtualbox/virtualbox.git/commit/?id=68f57408199e304df63285910a360bc2f6ae2372
> > 
> > So after that commit, a normal Depends from virtualbox to virtualbox
> > -dkms should be all that's nessesary.
> 
> thanks for the useful pointer, I admit I get headache when I have to deal 
> with such things :)
> 
> however, according to the debdiff of the fix:
[...]
> prior the virtualbox-dkms | virtualbox-source was in Recommends, not in 
> Depends.
> 
> was it really a circular dependency?

No sorry I meant that if you had added a normal Depends without the
above commit, you would have created a circular dependency.

Before these changes, the dependency from virtualbox-dkms to virtualbox
was causing virtualbox to always be configured first.

James

signature.asc
Description: This is a digitally signed message part

Re: promoting virtualbox-dkms to virtualbox pre-depends

[Gianfranco Costamagna]

Hi  James!


>This should work with a normal Depends relation (reread Policy 7.2).
>
>This commit helped since it prevented a circular dependency involving
>the two packages:
>https://anonscm.debian.org/cgit/pkg-virtualbox/virtualbox.git/commit/?id=68f57408199e304df63285910a360bc2f6ae2372
>
>So after that commit, a normal Depends from virtualbox to virtualbox
>-dkms should be all that's nessesary.



thanks for the useful pointer, I admit I get headache when I have to deal with 
such things :)

however, according to the debdiff of the fix:
@@ -87,11 +87,9 @@


Package: virtualbox
Architecture: amd64 i386
-Pre-Depends: dpkg (>= 1.15.6~)
+Pre-Depends: dpkg (>= 1.15.6~), virtualbox-dkms (= ${source:Version}) | 
virtualbox-source (= ${source:Version})
Depends: adduser, ${misc:Depends}, ${python:Depends}, ${shlibs:Depends}
-Recommends: virtualbox-dkms (= ${source:Version}) | virtualbox-source (= 
${source:Version}),
-virtualbox-qt (= ${binary:Version}),
-${shlibs:Recommends}
+Recommends: virtualbox-qt (= ${binary:Version}), ${shlibs:Recommends}
Suggests: vde2, virtualbox-guest-additions-iso
Conflicts: virtualbox-2.0,
virtualbox-2.1,
@@ -132,7 +130,8 @@
Section: contrib/kernel
Architecture: all
Pre-Depends: dpkg (>= 1.15.6~)
-Depends: virtualbox (>= ${source:Version}), ${misc:Depends}
+Depends: ${misc:Depends}
+Recommends: virtualbox (>= ${source:Version})
Description: x86 virtualization solution - kernel module sources for dkms
VirtualBox is a free x86 virtualization solution allowing a wide range
of x86 operating systems such as Windows, DOS, BSD or Linux to run on a


prior the virtualbox-dkms | virtualbox-source was in Recommends, not in Depends.

was it really a circular dependency?

Now the package works, so I would like to avoid breaking it again :)

cheers,

Gianfranco

promoting virtualbox-dkms to virtualbox pre-depends

[Gianfranco Costamagna]

As shown in policy 7.2

"You should not specify a Pre-Depends entry for a package before this has been 
discussed on the debian-devel mailing
list and a consensus about doing that has been reached. See Dependencies, 
Section 3.5."

the problem actually is that virtualbox-dkms should be configured *before* 
configuring virtualbox, otherwise
without a built kernel module, restarting VMs
will fail and lead to an half-configured package.

Please see bugs #798527 and #798979 as examples.

(this is a subpackage depending on another subpackage that belongs to the same 
src, I don't get why I should discuss such internal
changes, but well, policy is policy)



cheers,

Gianfranco

Re: promoting virtualbox-dkms to virtualbox pre-depends

[James Cowgill]

Hi,

On Tue, 2015-09-22 at 15:00 +, Gianfranco Costamagna wrote:
> As shown in policy 7.2
> 
> "You should not specify a Pre-Depends entry for a package before this
> has been discussed on the debian-devel mailing
> list and a consensus about doing that has been reached. See
> Dependencies, Section 3.5."
> 
> the problem actually is that virtualbox-dkms should be configured
> *before* configuring virtualbox, otherwise
> without a built kernel module, restarting VMs
> will fail and lead to an half-configured package.
> 
> Please see bugs #798527 and #798979 as examples.

This should work with a normal Depends relation (reread Policy 7.2).

This commit helped since it prevented a circular dependency involving
the two packages:
https://anonscm.debian.org/cgit/pkg-virtualbox/virtualbox.git/commit/?id=68f57408199e304df63285910a360bc2f6ae2372

So after that commit, a normal Depends from virtualbox to virtualbox
-dkms should be all that's nessesary.

Thanks,
James

signature.asc
Description: This is a digitally signed message part