Re: DEP 17: Improve support for directory aliasing in dpkg

[Luca Boccassi]

On Fri, 28 Apr 2023 at 10:12, Luca Boccassi  wrote:
>
> On Fri, 28 Apr 2023 at 09:09, Helmut Grohne  wrote:
> > So yeah, with the exception of dash, this looks fairly good. Let me also
> > dive into dash. Unlike the majority of diverters, it diverts in postinst
> > rather than preinst to allow controlling /bin/sh via debconf. A similar
> > technique is in effect by gpr. In any case, this is special, because
> > dash diverts its own files, so when moving dash's file, its diversions
> > can be migrated at the same time. It merely means, that we cannot have
> > debhelper just move files (as that would horribly break dash) and
> > instead have to move files on a package-by-package way. We could also
> > opt for removing dash's diversion in the default case and there even is
> > a patch for doing so (#989632) since almost two years. Too bad we didn't
> > apply it. In any case, as long as the file moving is not forced via
> > debhelper, dash should be harmless.
>
> If I understand correctly, by "forced via debhelper" you mean the
> proposal of fixing the paths at build time, right? But if not via
> that, it means having to fix all of them by hand, which is a lot - is
> it possible to fix dash instead? Or else, we could add an opt-out via
> one of the usual dh mechanisms, and use it only in dash perhaps?

Also as pointed out by the maintainer the debconf machinery was
dropped from dash, last year:
https://salsa.debian.org/debian/dash/-/commit/c322a1c9fc6be11d7eb4439

This should hopefully simplify things?

Kind regards,
Luca Boccassi

Re: DEP 17: Improve support for directory aliasing in dpkg

[Helmut Grohne]

Please excuse the volume of mails I am producing at this time, but I
still think this adds to the discussion.

On Thu, Apr 27, 2023 at 12:34:06AM +0200, Helmut Grohne wrote:
> I have a bad feeling about this. I think some dpkg maintainer warned us
> that diversions would break. Let's peek at his list again. He also said
> update-alternatives would be broken. I admit not having dug into this
> yet, but my gut feeling already is that update-alternatives will become
> "funny" as well though I guess we cannot fix update-alternatives by
> adding alternatives.

Sometimes, feelings can be wrong. I admit not having done much with
alternatives, so let me roughly summarize the relevant aspects and then
evaluate the aliasing effects.

Every alternative has an "alternative name". This name is the one inside
/etc/alternatives. When it refers to a binary in $PATH, usually that
binary name also is the name of the alternative.  Consequently, it is
very unlikely for us to have a conflict on this name due to aliasing.

Every alternative also has "generic name", which is the primary path of
the symlink installed. This can be /bin/something or /usr/bin/something
or something else entirely. I looked through the various paths I could
find in alternatives and never found two distinct alternatives using
aliased paths, so this much will very likely go without problems.

I looked around for possible non-canonical paths in alternatives (by
installing lots of packages and discovered these:
 * /bin/bsd-csh is shipped by csh and is provided as primary target for
   the csh alternative
 * /bin/csh is a generic name for the csh alternative
 * /bin/ed is shipped by ed and is provided as primary target for the
   editor alternative
 * /bin/elvis-tiny is shipped by elvis-tiny and is provided as primary
   target for the editor alternative
 * /bin/more is shipped by util-linux and is provided as primary target
   for the pager alternative
 * /bin/ksh is a generic name for the ksh alternative
 * /bin/ksh93 is shipped by ksh93u+m and is provided as primary target
   for the ksh alternative
 * /bin/mksh is shipped by mksh and is provided both as primary and as
   slave target for the ksh alternative
 * /bin/mksh-static is shipped by mksh and is provided both as primary
   and as slave target for the ksh alternative
 * /bin/mt is a generic name for the mt alternative
 * /bin/mt-gnu is shipped by cpio and is provided as primary target
   for the mt alternative
 * /bin/mt-st is shipped by mt-st and is provided as primary target
   for the mt alternative
 * /bin/nano is shipped by nano and is provided as primary target
   for the editor alternative
 * /bin/nano-tiny is shipped by nano-tiny and is provided as primary
   target for the editor alternative
 * /bin/nc is a generic name for the nc alternative
 * /bin/nc.openbsd is shipped by netcat-openbsd and is provided as
   primary target and slave target for the nc alternative
 * /bin/nc.traditional is shipped by netcat-traditional and is provided
   as primary target and slave target for the nc alternative
 * /bin/netcat is a slave link for the nc alternative
 * /bin/rc is a generic name for the rc alternative and installed by
   9base
 * /bin/rksh is a slave link for the ksh alternative
 * /bin/rksh93 is shipped by ksh93u+m and is provided as slave target
   for the ksh alternative
 * /bin/tcsh is shipped by tcsh and is provided as a primary target for
   the csh alternative
 * /bin/true is not shipped by uim, but installed as primary target for
   the uim-toolbar alternative
 * /lib/cpp is a generic name for the cpp alternative and installed by
   cpp
 * /lib/systemd/system/ipset.service is a generic name for the
   ipset.service alternative
 * /lib/systemd/system/iptables.service is a generic name for the
   iptables.service alternative
 * /lib/systemd/system/netfilter-persistent.service is shipped by
   iptables-persistent and provided as primary target and slave target
   for the iptables.service and ipset.service alternatives by
   iptables-persistent and ipset-persistent respectively
 * /lib/systemd/system/ip6tables.service is a slave link for the
   iptables.service alternative
 * /lib/firmware/regulatory.db is a generic name for the regulartory.db
   alternative
 * /lib/firmware/regulatory.db.p7s is a slave link for the
   regulatory.db alternative
 * /lib/firmware/regulatory.db.p7s-debian is shipped by wireless-regdb
   and is provided as slave target for the regulatory.db alternative
 * /lib/firmware/regulatory.db.p7s-upstream is shipped by wireless-regdb
   and is provided as slave target for the regulatory.db alternative
 * /lib/firmware/regulatory.db-debian is shipped by wireless-regdb and
   is provided as primary alternative for the regulatory.db alternative
 * /lib/firmware/regulatory.db-upstream is shipped by wireless-regdb and
   is provided as primary alternative for the regulatory.db alternative

I think ideally, we'd want these to become canonicalized eventually. And
this is w

Re: DEP 17: Improve support for directory aliasing in dpkg

[Russ Allbery]

Helmut Grohne  writes:
> On Thu, Apr 27, 2023 at 10:58:46AM +0200, Marc Haber wrote:

>> My gut feeling is that we are wasting prescious time of numerous
>> skilled Debian Developers to find ugly workarounds to something that
>> should be done in dpkg, but isnt being done because one dpkg maintainer
>> has decided to not go the way the project has decided to go.

> I find this mail of yours very disappointing and possibly even failing
> our Code of Conduct on multiple accounts.

I am unhappy to see the Code of Conduct used in this way.

Marc's message was not a personal attack.  It did not assume bad faith, or
indeed make any statements about motives at all.  He expressed his opinion
about project priorities and put it in the context of his personal
judgment of the facts of the situation as he sees them.

You may disagree with his summary of facts, or his opinion about or
evaluation of the current situation, or even the usefulness of him raising
this point due to lack of resources.  It is certainly appropriate to raise
those disagreements in response, or even to ignore the message if you
don't think it's a constructive line of discussion.  (In particular, I
think Marc assumes that a solution in dpkg would be more straightforward,
something that I think is debatable on technical grounds.)

But to say that this is possibly a violation of the Code of Conduct is to
say that this message doesn't meet the bar for civil discussion on our
lists, and I think it is unreasonable to expect anyone to be more civil or
even-handed than Marc was in his summary of behavior that he strongly
disagrees with.  (And, to state the obvious, I don't believe that message
was a violation of our Code of Conduct.)  Trying to set the bar higher
than this would have the effect of forbidding particular types of hard
conversations, which is not healthy for the project.

We have to be able to talk about interpersonal disagreements and problems
of alignment of motives and goals among the people working on the project.
Sometimes those discussions are going to be uncomfortable, but we can't
ignore them and never discuss them because they're uncomfortable.  We are
a collection of humans working together collaboratively, which means there
will be tension and conflict and we have to deal with that, constructively
but honestly and forthrightly.

Part of working collaboratively with other people is that those people get
to criticize how you are doing your work, as long as they do so
respectfully and assuming good faith.  Sometimes that includes saying that
one believes the actions of another developer are causing a misallocation
of project resources or time.  Whether or not we end up agreeing that is
true, this is a valid topic for discussion, and sometimes it is feedback
that other developers need to hear so that they can do some introspection
and evaluate whether that may indeed be the case.

-- 
Russ Allbery (r...@debian.org)  

Re: DEP 17: Improve support for directory aliasing in dpkg

[James Addison]

On Fri, 28 Apr 2023 at 17:52, Jochen Sprickerhof  wrote:
>
> Hi James,
>
> * James Addison  [2023-04-28 14:54]:
> >To make sure we don't miss any packages out accidentally: could you
> >confirm that those hundred-or-so errors occurred from 27 or so
> >distinct packages?
> >
> >(looking at RC bugs created within the past week, I currently find 27
> >bugs with 'Breaks+Replaces' in the title)
> >
> >https://udd.debian.org/bugs.cgi?release=na&merged=ign&keypackages=only&fnewer=only&fnewerval=7&flastmodval=7&rc=1&cpopcon=1&chints=1&ckeypackage=1&ctags=1&cdeferred=1&caffected=1&sortby=last_modified&sorto=asc&format=html#results
>
> That's only key packages. Here is the full list:
>
> https://bugs.debian.org/cgi-bin/pkgreport.cgi?dist=unstable;include=subject%3Amissing+Breaks%2BReplaces;submitter=helmut%40subdivi.de
>
> Cheers Jochen

Ah, typical user error from me.  Anyway - glad to find that they've
all been filed.  Thank you, Jochen!

Re: DEP 17: Improve support for directory aliasing in dpkg

[Jochen Sprickerhof]

Hi James,

* James Addison  [2023-04-28 14:54]:

To make sure we don't miss any packages out accidentally: could you
confirm that those hundred-or-so errors occurred from 27 or so
distinct packages?

(looking at RC bugs created within the past week, I currently find 27
bugs with 'Breaks+Replaces' in the title)

https://udd.debian.org/bugs.cgi?release=na&merged=ign&keypackages=only&fnewer=only&fnewerval=7&flastmodval=7&rc=1&cpopcon=1&chints=1&ckeypackage=1&ctags=1&cdeferred=1&caffected=1&sortby=last_modified&sorto=asc&format=html#results


That's only key packages. Here is the full list:

https://bugs.debian.org/cgi-bin/pkgreport.cgi?dist=unstable;include=subject%3Amissing+Breaks%2BReplaces;submitter=helmut%40subdivi.de

Cheers Jochen


signature.asc
Description: PGP signature

Re: DEP 17: Improve support for directory aliasing in dpkg

[James Addison]

On Thu, 27 Apr 2023 at 14:37, Helmut Grohne  wrote:
>
> Hi Simon,
>
> On Sat, Apr 22, 2023 at 11:41:29AM +0100, Simon McVittie wrote:
> > You might reasonably say that "the maintainer of bar didn't add the
> > correct Breaks/Replaces on foo" is a RC bug in bar - and it is! - but
> > judging by the number of "missing Breaks/Replaces" bug reports that have
> > to be opened by unstable users (sometimes me), it's a very easy mistake
> > to make.
>
> That number seemed quite vague to me and I wanted to get a better handle
> on it. The rough idea here should be that we have some package from
> bullseye and "upgrade" it to a different package from bookworm.
> Generating useful candidates for this can be done using Contents. Given
> candidates, I've attached a validation script:
>
> ./check_conflicts.sh $OLDPKG bullseye $NEWPKG bookworm
>
> In order to draw value from it, the output must be parsed. The exit code
> can be non-zero for various reasons. As for candidate generation, I
> think one can either just try them all (which takes a bit longer on the
> validation phase) or reduce their number by ignoring existing
> Breaks+Replaces, but I haven't found an elegant solution for the latter
> yet.
>
> In any case, unstable has around:
>  * 5700 Breaks
>  * 6500 Replaces
>  * 100 unpack errors due to missing Breaks+Replaces
>
> That latter number has just been turned into rc bugs...

Hi Helmut,

To make sure we don't miss any packages out accidentally: could you
confirm that those hundred-or-so errors occurred from 27 or so
distinct packages?

(looking at RC bugs created within the past week, I currently find 27
bugs with 'Breaks+Replaces' in the title)

https://udd.debian.org/bugs.cgi?release=na&merged=ign&keypackages=only&fnewer=only&fnewerval=7&flastmodval=7&rc=1&cpopcon=1&chints=1&ckeypackage=1&ctags=1&cdeferred=1&caffected=1&sortby=last_modified&sorto=asc&format=html#results

Thanks,
James

Re: DEP 17: Improve support for directory aliasing in dpkg

[Helmut Grohne]

Hi Simon,

On Fri, Apr 28, 2023 at 02:07:33PM +0200, Simon Richter wrote:
> Transforming existing diversions: yes, if you can find out about them
> without looking at dpkg internal files. It may very well be necessary to
> update the file format on one of these, and if that would cause your
> script to create nonsense diversions, that'd be another thing we'd have
> to work around while building real aliasing support.
> 
> My current mood is "I'd rather focus on a proper solution, not another
> hack that needs to be supported by the proper solution."
> 
> Anything we build here that is not aliasing support for dpkg, but
> another "shortcut" will delay aliasing support for dpkg because it adds
> more possible code paths that all need to be tested.
> 
> Keep in mind that we're here because someone took a shortcut, after all.

I think we have a misunderstanding here. As far as I understand it, the
core idea of Luca's approach is that we move all files to their
canonical locations and then - when nothing is left in directories such
as /bin or /lib - there is no aliasing anymore, which is why we do not
have to teach dpkg about aliasing and never patch it.

>From my point of view the only reason to try and solve this with a pile
of hacks is get us to a state that the current dpkg can deal well with
again (because all aliasing is gone). And while I've argued earlier that
dpkg will need to support aliasing, I'm trying to get myself convinced
that this is not necessary.  Personally, I don't have a final conclusion
on this yet. Can I ask you to go into more detail as to why you think
that patching dpkg is ultimately necessary?

At the point where we conclude that no, we cannot move forward without
patching dpkg, I fully agree with you that taking more shortcuts is only
going to make it worse.

Please do understand my research as evaluating all possible approaches
(and their consequences) in parallel. I'm not trying to push a
particular approach other than trying to move the whole matter forward.
Once we have a better understanding, we'll have to build consensus
around one of these approaches somehow.

Helmut

Bug#1035055: ITP: firmware-imx -- Firmware binary blobs needed on NXP i.MX platforms

[Manuel Traut]

Package: wnpp
Severity: wishlist
Owner: Manuel Traut 
X-Debbugs-Cc: debian-devel@lists.debian.org, manuel.tr...@mt.com, 
ma...@mecka.net

* Package name: firmware-imx
  Version : 8.19
  Upstream Contact: NXP
* URL : https://www.nxp.com/
* License : LA_OPT_NXP_Software_License_v42
  Description : Firmware binary blobs needed on NXP i.MX platforms

i.MX Firmware including firmware for VPU, DDR, EPDC, HDMI, DP
(Display Port), and SDMA

To build a working u-boot for i.MX8* at least the DDR train binaries
from NXP are needed.

More details are available in this document:
https://www.nxp.com/docs/en/release-note/IMX_LINUX_RELEASE_NOTES.pdf

The package shall provide the files from firmware-imx-8.19.bin:

epdc/epdc_ED060XH2C1.fw.nonrestricted
epdc/epdc_ED060XH7U2.fw
epdc/epdc_E97_V110.fw
epdc/epdc_E60_V220.fw
epdc/epdc_E60_V110.fw
epdc/epdc_E060SCM.fw
sdma/sdma-imx7d.bin
sdma/sdma-imx51-to3.bin
sdma/sdma-imx31-to2.bin
sdma/sdma-imx35-to2.bin
sdma/sdma-imx35-to1.bin
sdma/sdma-imx25-to1.bin
sdma/sdma-imx6q.bin
sdma/sdma-imx31-to1.bin
sdma/sdma-imx53-to1.bin
xuvi/vpu_fw_imx8_xuvi.bin
ddr/synopsys/ddr4_imem_1d.bin
ddr/synopsys/ddr3_dmem_1d_201810.bin
ddr/synopsys/ddr4_dmem_2d_201810.bin
ddr/synopsys/lpddr4_pmu_train_2d_dmem_202006.bin
ddr/synopsys/lpddr4_pmu_train_1d_dmem_202006.bin
ddr/synopsys/lpddr4_pmu_train_2d_imem_202006.bin
ddr/synopsys/lpddr4_pmu_train_1d_imem.bin
ddr/synopsys/ddr4_imem_2d.bin
ddr/synopsys/ddr4_imem_1d_202006.bin
ddr/synopsys/lpddr4_dmem_1d_v202201.bin
ddr/synopsys/lpddr4_pmu_train_2d_imem_201904.bin
ddr/synopsys/ddr4_dmem_1d_201810.bin
ddr/synopsys/lpddr4_pmu_train_1d_dmem_201904.bin
ddr/synopsys/ddr4_dmem_2d.bin
ddr/synopsys/lpddr4_pmu_train_2d_dmem.bin
ddr/synopsys/lpddr4_pmu_train_1d_imem_202006.bin
ddr/synopsys/ddr4_dmem_1d_202006.bin
ddr/synopsys/ddr4_dmem_1d.bin
ddr/synopsys/lpddr4_pmu_train_1d_dmem.bin
ddr/synopsys/ddr4_imem_1d_201810.bin
ddr/synopsys/lpddr4_pmu_train_2d_dmem_201904.bin
ddr/synopsys/lpddr4_imem_1d_v202201.bin
ddr/synopsys/ddr4_imem_2d_202006.bin
ddr/synopsys/ddr4_imem_2d_201810.bin
ddr/synopsys/ddr3_imem_1d_201810.bin
ddr/synopsys/lpddr4_imem_2d_v202201.bin
ddr/synopsys/ddr3_dmem_1d.bin
ddr/synopsys/ddr4_dmem_2d_202006.bin
ddr/synopsys/lpddr4_dmem_2d_v202201.bin
ddr/synopsys/lpddr4_pmu_train_1d_imem_201904.bin
ddr/synopsys/ddr3_imem_1d.bin
ddr/synopsys/lpddr4_pmu_train_2d_imem.bin
easrc/easrc-imx8mn.bin
vpu/vpu_fw_imx8_enc.bin
vpu/vpu_fw_imx8_dec.bin
vpu/vpu_fw_imx6q.bin
vpu/vpu_fw_imx53.bin
vpu/vpu_fw_imx27_TO1.bin
vpu/vpu_fw_imx51.bin
vpu/vpu_fw_imx6d.bin
vpu/vpu_fw_imx27_TO2.bin
hdmi/cadence/dp_ls1028a.bin
hdmi/cadence/signed_dp_imx8m.bin
hdmi/cadence/signed_hdmi_imx8m.bin
hdmi/cadence/hdmitxfw.bin
hdmi/cadence/hdmirxfw.bin
hdmi/cadence/dpfw.bin
xcvr/xcvr-imx8mp.bin

e.g. in /lib/firmware/imx

Bug#1035054: ITP: node-react-redux -- Official ReactJS bindings for node-redux

[Yadd]

Package: wnpp
Severity: wishlist
Owner: Yadd 
X-Debbugs-Cc: debian-devel@lists.debian.org

* Package name: node-react-redux
  Version : 8.0.5
  Upstream Contact: https://github.com/reduxjs/react-redux/issues
* URL : https://github.com/reduxjs/react-redux
* License : Expat
  Programming Lang: JavaScript
  Description : Official ReactJS bindings for node-redux

node-react is a library for building JavaScript user interfaces.
node-redux is a predictable state container for JavaScript apps.
This package provides the official bindings between these modules.

It's a dependency of JupyterLab, it'll be maintained under JS Team
umbrella.

Re: DEP 17: Improve support for directory aliasing in dpkg

[Simon Richter]

Hi,

On Thu, Apr 27, 2023 at 12:34:06AM +0200, Helmut Grohne wrote:

> Ok, let's move on. I've proposed diversions as a cure, but in reality
> diversions are a problem themselves. Consider that
> cryptsetup-nuke-password diverts /lib/cryptsetup/askpass, which is
> usually owned by cryptsetup. If cryptsetup were to move that file to
> /usr, the diversion would not cover it anymore and the actual content of
> askpass would depend on the unpack order. That's very bad and none of
> what I proposed earlier is going to fix this.

Another question: how would this interact with a patch that teaches dpkg
to do aliases properly, because such a patch would affect diversion
handling as well -- specifically, aliases mean that diversions are also
aliased (I believe my patch implicitly does this right, but I think I
just got 60 more testcases to write), and that diversion targets are
implicitly redirected to a resolved form (I don't do that yet, but it's
simple to add to my patch).

I think the "divert, but do not rename" approach itself should be fairly
safe, because all it does is make a deletion fail. Registering the
diversion to the new package should be sufficient to make sure the newly
unpacked file is not diverted. This probably needs some additional undo
operations for failed installs/upgrades, so the diversion is properly
removed in these cases (there is no guarantee that postinst will be
called after preinst, we could also end up in postrm).

> So how do we fix diversions? Let's have a look into the dpkg toolbox
> again. I've got an idea. Diversions. What you say? How do you fix
> diversions with diversions? Quite obviously, you divert
> /usr/bin/dpkg-divert! And whenever dpkg-divert is instructed to add a
> diversion for a non-canonical path, you forward that call to the real
> dpkg-divert, but also call it with a canonicalized version such that
> both locations are covered. When initially deploying the diversion of
> /usr/bin/dpkg-divert, we also need to transform existing diversions.

Ouch, if you deploy that, I will definitely need to add diversion
merging code to alias registration. That's another 60 testcases, but we
need defined behaviour for that anyway.

Transforming existing diversions: yes, if you can find out about them
without looking at dpkg internal files. It may very well be necessary to
update the file format on one of these, and if that would cause your
script to create nonsense diversions, that'd be another thing we'd have
to work around while building real aliasing support.

My current mood is "I'd rather focus on a proper solution, not another
hack that needs to be supported by the proper solution."

Anything we build here that is not aliasing support for dpkg, but
another "shortcut" will delay aliasing support for dpkg because it adds
more possible code paths that all need to be tested.

Keep in mind that we're here because someone took a shortcut, after all.

   Simon

Bug#1035050: ITP: node-rjsf -- ReactJS component capable of using JSON Schema to declaratively build forms

[Yadd]

Package: wnpp
Severity: wishlist
Owner: Yadd 
X-Debbugs-Cc: debian-devel@lists.debian.org

* Package name: node-rjsf
  Version : 5.6.2
  Upstream Contact:
  https://github.com/rjsf-team/react-jsonschema-form/issues
* URL : https://github.com/rjsf-team/react-jsonschema-form
* License : Apache-2.0
  Programming Lang: JavaScript
  Description : ReactJS component capable of using JSON Schema to 
declaratively build forms

node-rjsf provides ReactJS components capable of using JSON Schema to
declaratively build and customize web forms.

It's a dependency of JupyterLab and will be maintained under JS Team
umbrella

Bug#1035049: ITP: nfstest -- NFS Test Suite

[Gürkan Myczko]

Package: wnpp
Severity: wishlist
Owner: Gürkan Myczko 
X-Debbugs-Cc: debian-devel@lists.debian.org

* Package name: nfstest
  Version : 3.2
  Upstream Authors: Jorge Mora 
  URL : https://wiki.linux-nfs.org/wiki/index.php/NFStest
* License : GPL-2+
  Description : NFS Test Suite
 This is a set of tools for testing either the NFS client or the NFS 
server,

 included tests focused mainly on testing the client.

Re: DEP 17: Improve support for directory aliasing in dpkg

[Timo Röhling]

* Luca Boccassi  [2023-04-28 10:12]:

Which part of config-package-dev causes a conflict here? Is it
something that can be fixed? Given it's declarative, an upload + rdeps
rebuild should be all that's needed, assuming we know what the issue
is and how to fix it. As far as I can remember, it's a build-time
utility and everything it does is embedded in the target package's
maintainer scripts. But it's been a few years since I last used it, so
I might remember wrongly.


You remember correctly. It is relatively straight-forward to patch
config-package-dev to create additional diversions for files in /
and /usr (if we decide that is the way forward), but admins will
have to rebuild their local config packages for these changes to
take effect.


Cheers
Timo

--
⢀⣴⠾⠻⢶⣦⠀   ╭╮
⣾⠁⢠⠒⠀⣿⡁   │ Timo Röhling   │
⢿⡄⠘⠷⠚⠋⠀   │ 9B03 EBB9 8300 DF97 C2B1  23BF CC8C 6BDD 1403 F4CA │
⠈⠳⣄   ╰╯


signature.asc
Description: PGP signature

Re: RFP: kernelcraft -- Tool to build and run a kernel inside a virtualized snapshot of your live system

[Andrea Righi]

On Thu, Apr 27, 2023 at 05:56:41PM +0200, Andrea Righi wrote:
> Package: wnpp
> Severity: wishlist
> Owner: Andrea Righi 
> X-Debbugs-CC: debian-devel@lists.debian.org
> Control: affects -1 ITP
> 
> * Package name: kernelcraft
>   Version : 2.0
>   Upstream Author : Andrea Righi 
> * URL : https://salsa.debian.org/arighi/kernelcraft
> * License : GPL-2
>   Programming Lang: Python
>   Description : Tool to build and run a kernel inside a virtualized 
> snapshot of your live system
> 
> KernelCraft is a tool that allows to easily and quickly recompile and
> test a Linux kernel, starting from the source code.
> 
> It allows to recompile the kernel in few minutes (rather than hours),
> then the kernel is automatically started in a virtualized environment
> that is an exact copy-on-write copy of your live system, which means
> that any changes made to the virtualized environment do not affect the
> host system.
> 
> In order to do this a minimal config is produced (with the bare minimum
> support to test the kernel inside qemu), then the selected kernel is
> automatically built and started inside qemu, using the filesystem of the
> host as a copy-on-write snapshot.
> 
> This means that you can safely destroy the entire filesystem, crash the
> kernel, etc. without affecting the host.
> 
> Kernels produced with KernelCraft are lacking lots of features, in order
> to reduce the build time to the minimum and still provide you a usable
> kernel capable of running your tests and experiments.
> 
> KernelCraft is based on virtme, written by Andy Lutomirski
> (https://git.kernel.org/pub/scm/utils/kernel/virtme/virtme.git).

Please ignore this, we are currectly renaming the project to virtme-ng
to avoid a potential naming conflict with another project.

I will send another RFP when the renaming will be completed.

Sorry for the noise,
-Andrea

Re: DEP 17: Improve support for directory aliasing in dpkg

[Luca Boccassi]

On Fri, 28 Apr 2023 at 09:09, Helmut Grohne  wrote:
>
> On Thu, Apr 27, 2023 at 12:34:06AM +0200, Helmut Grohne wrote:
> > Ok, let's move on. I've proposed diversions as a cure, but in reality
> > diversions are a problem themselves. Consider that
> > cryptsetup-nuke-password diverts /lib/cryptsetup/askpass, which is
> > usually owned by cryptsetup. If cryptsetup were to move that file to
> > /usr, the diversion would not cover it anymore and the actual content of
> > askpass would depend on the unpack order. That's very bad and none of
> > what I proposed earlier is going to fix this.
> >
> > And of course, this is not some special example, it's a pattern:
> >  * /lib/udev/rules.d/60-cdrom_id.rules: udev -> amazon-ec2-utils
> >  * /sbin/dhclient: isc-dhcp-client -> isc-dhcp-client-ddns
> >  * /bin/systemd-sysusers: systemd -> opensysusers
> >  * ...
> >
> > So how do we fix diversions? Let's have a look into the dpkg toolbox
> > again. I've got an idea. Diversions. What you say? How do you fix
> > diversions with diversions? Quite obviously, you divert
> > /usr/bin/dpkg-divert! And whenever dpkg-divert is instructed to add a
> > diversion for a non-canonical path, you forward that call to the real
> > dpkg-divert, but also call it with a canonicalized version such that
> > both locations are covered. When initially deploying the diversion of
> > /usr/bin/dpkg-divert, we also need to transform existing diversions.
> > Other than that, things should work after doubling down on diversions.
> > Sorry, I don't have a test case for this yet.
>
> I still don't have a test case, but I have data. Using
> binarycontrol.d.n, I identified packages setting up diversions in
> preinst (this seems most common, but dash for instance sets up a
> diversion in postinst instead, so there are some false negatives). And
> while I initially tried to parse those preinst scripts, solving the
> halting problem seemed just too hard, so I opted for just running them.
> I'm attaching the relevant scripts and showing the affected diversions:
>
> diversion of /lib/udev/rules.d/60-cdrom_id.rules to 
> /lib/udev/rules.d/60-cdrom_id.rules.disabled by amazon-ec2-utils in stable, 
> testing, unstable
> diversion of /sbin/coldreboot to /lib/container/divert/coldreboot.orig by 
> bfh-container in testing, unstable
> diversion of /sbin/halt to /lib/container/divert/halt.orig by bfh-container 
> in testing, unstable
> diversion of /sbin/poweroff to /lib/container/divert/poweroff.orig by 
> bfh-container in testing, unstable
> diversion of /sbin/reboot to /lib/container/divert/reboot.orig by 
> bfh-container in testing, unstable
> diversion of /sbin/shutdown to /lib/container/divert/shutdown.orig by 
> bfh-container in testing, unstable
> diversion of /lib/cryptsetup/askpass to /lib/cryptsetup/askpass.cryptsetup by 
> cryptsetup-nuke-password in testing, unstable
> diversion of /sbin/dhclient to /sbin/dhclient-noddns by isc-dhcp-client-ddns 
> in stable, testing, unstable
> diversion of /sbin/coldreboot to /lib/molly-guard/coldreboot by molly-guard 
> in stable, testing, unstable
> diversion of /sbin/halt to /lib/molly-guard/halt by molly-guard in stable, 
> testing, unstable
> diversion of /sbin/poweroff to /lib/molly-guard/poweroff by molly-guard in 
> stable, testing, unstable
> diversion of /sbin/reboot to /lib/molly-guard/reboot by molly-guard in 
> stable, testing, unstable
> diversion of /sbin/shutdown to /lib/molly-guard/shutdown by molly-guard in 
> stable, testing, unstable
> diversion of /bin/systemd-sysusers to /bin/systemd-sysusers.real by 
> opensysusers in stable, testing, unstable
> diversion of /sbin/coldreboot to 
> /lib/open-infrastructure/container/divert/coldreboot.orig by 
> progress-linux-container in stable, testing, unstable
> diversion of /sbin/halt to 
> /lib/open-infrastructure/container/divert/halt.orig by 
> progress-linux-container in stable, testing, unstable
> diversion of /sbin/poweroff to 
> /lib/open-infrastructure/container/divert/poweroff.orig by 
> progress-linux-container in stable, testing, unstable
> diversion of /sbin/reboot to 
> /lib/open-infrastructure/container/divert/reboot.orig by 
> progress-linux-container in stable, testing, unstable
> diversion of /sbin/shutdown to 
> /lib/open-infrastructure/container/divert/shutdown.orig by 
> progress-linux-container in stable, testing, unstable
> diversion of /bin/zcat to /bin/zcat.gzip by zutils in stable, testing, 
> unstable
> diversion of /bin/zcmp to /bin/zcmp.gzip by zutils in stable, testing, 
> unstable
> diversion of /bin/zdiff to /bin/zdiff.gzip by zutils in stable, testing, 
> unstable
> diversion of /bin/zegrep to /bin/zegrep.gzip by zutils in stable, testing, 
> unstable
> diversion of /bin/zfgrep to /bin/zfgrep.gzip by zutils in stable, testing, 
> unstable
> diversion of /bin/zgrep to /bin/zgrep.gzip by zutils in stable, testing, 
> unstable
>
> All other diversion affect /etc or /usr and I think we're not going to
> move any files from /usr 

Re: DEP 17: Improve support for directory aliasing in dpkg

[Helmut Grohne]

On Thu, Apr 27, 2023 at 12:34:06AM +0200, Helmut Grohne wrote:
> Ok, let's move on. I've proposed diversions as a cure, but in reality
> diversions are a problem themselves. Consider that
> cryptsetup-nuke-password diverts /lib/cryptsetup/askpass, which is
> usually owned by cryptsetup. If cryptsetup were to move that file to
> /usr, the diversion would not cover it anymore and the actual content of
> askpass would depend on the unpack order. That's very bad and none of
> what I proposed earlier is going to fix this.
> 
> And of course, this is not some special example, it's a pattern:
>  * /lib/udev/rules.d/60-cdrom_id.rules: udev -> amazon-ec2-utils
>  * /sbin/dhclient: isc-dhcp-client -> isc-dhcp-client-ddns
>  * /bin/systemd-sysusers: systemd -> opensysusers
>  * ...
> 
> So how do we fix diversions? Let's have a look into the dpkg toolbox
> again. I've got an idea. Diversions. What you say? How do you fix
> diversions with diversions? Quite obviously, you divert
> /usr/bin/dpkg-divert! And whenever dpkg-divert is instructed to add a
> diversion for a non-canonical path, you forward that call to the real
> dpkg-divert, but also call it with a canonicalized version such that
> both locations are covered. When initially deploying the diversion of
> /usr/bin/dpkg-divert, we also need to transform existing diversions.
> Other than that, things should work after doubling down on diversions.
> Sorry, I don't have a test case for this yet.

I still don't have a test case, but I have data. Using
binarycontrol.d.n, I identified packages setting up diversions in
preinst (this seems most common, but dash for instance sets up a
diversion in postinst instead, so there are some false negatives). And
while I initially tried to parse those preinst scripts, solving the
halting problem seemed just too hard, so I opted for just running them.
I'm attaching the relevant scripts and showing the affected diversions:

diversion of /lib/udev/rules.d/60-cdrom_id.rules to 
/lib/udev/rules.d/60-cdrom_id.rules.disabled by amazon-ec2-utils in stable, 
testing, unstable
diversion of /sbin/coldreboot to /lib/container/divert/coldreboot.orig by 
bfh-container in testing, unstable
diversion of /sbin/halt to /lib/container/divert/halt.orig by bfh-container in 
testing, unstable
diversion of /sbin/poweroff to /lib/container/divert/poweroff.orig by 
bfh-container in testing, unstable
diversion of /sbin/reboot to /lib/container/divert/reboot.orig by bfh-container 
in testing, unstable
diversion of /sbin/shutdown to /lib/container/divert/shutdown.orig by 
bfh-container in testing, unstable
diversion of /lib/cryptsetup/askpass to /lib/cryptsetup/askpass.cryptsetup by 
cryptsetup-nuke-password in testing, unstable
diversion of /sbin/dhclient to /sbin/dhclient-noddns by isc-dhcp-client-ddns in 
stable, testing, unstable
diversion of /sbin/coldreboot to /lib/molly-guard/coldreboot by molly-guard in 
stable, testing, unstable
diversion of /sbin/halt to /lib/molly-guard/halt by molly-guard in stable, 
testing, unstable
diversion of /sbin/poweroff to /lib/molly-guard/poweroff by molly-guard in 
stable, testing, unstable
diversion of /sbin/reboot to /lib/molly-guard/reboot by molly-guard in stable, 
testing, unstable
diversion of /sbin/shutdown to /lib/molly-guard/shutdown by molly-guard in 
stable, testing, unstable
diversion of /bin/systemd-sysusers to /bin/systemd-sysusers.real by 
opensysusers in stable, testing, unstable
diversion of /sbin/coldreboot to 
/lib/open-infrastructure/container/divert/coldreboot.orig by 
progress-linux-container in stable, testing, unstable
diversion of /sbin/halt to /lib/open-infrastructure/container/divert/halt.orig 
by progress-linux-container in stable, testing, unstable
diversion of /sbin/poweroff to 
/lib/open-infrastructure/container/divert/poweroff.orig by 
progress-linux-container in stable, testing, unstable
diversion of /sbin/reboot to 
/lib/open-infrastructure/container/divert/reboot.orig by 
progress-linux-container in stable, testing, unstable
diversion of /sbin/shutdown to 
/lib/open-infrastructure/container/divert/shutdown.orig by 
progress-linux-container in stable, testing, unstable
diversion of /bin/zcat to /bin/zcat.gzip by zutils in stable, testing, unstable
diversion of /bin/zcmp to /bin/zcmp.gzip by zutils in stable, testing, unstable
diversion of /bin/zdiff to /bin/zdiff.gzip by zutils in stable, testing, 
unstable
diversion of /bin/zegrep to /bin/zegrep.gzip by zutils in stable, testing, 
unstable
diversion of /bin/zfgrep to /bin/zfgrep.gzip by zutils in stable, testing, 
unstable
diversion of /bin/zgrep to /bin/zgrep.gzip by zutils in stable, testing, 
unstable

All other diversion affect /etc or /usr and I think we're not going to
move any files from /usr to /. So this is a complete list as of today
and I have to say, I expected it to be longer. In effect, we're talking
about merely 8 packages.

For completeness sake, I also looked at the other packages mentioning
dpkg-divert in their prei