Bug#1030841: ITP: frp -- A fast reverse proxy to help you expose a local server behind a NAT or firewall to the internet

[Bo YU]

Package: wnpp
Severity: wishlist
Owner: Bo YU 
X-Debbugs-Cc: debian-devel@lists.debian.org

* Package name: frp
  Version : 0.36.1-1
  Upstream Author :
* URL : https://github.com/fatedier/frp
* License : Apache-2.0
  Programming Lang: Go
  Description : A fast reverse proxy to help you expose a local server 
behind a NAT or firewall to the internet.

 .
 frp is a fast reverse proxy to help you expose a local server behind a
 NAT or firewall to the Internet. As of now, it supports **TCP** and
 **UDP**, as well as **HTTP** and **HTTPS** protocols, where requests can
 be forwarded to internal services by domain name.
 .

 This package will be maintained under Debian go team.



signature.asc
Description: PGP signature

Re: Help patching files in dependency package

[Dominic Hargreaves]

On Tue, Jan 17, 2023 at 08:52:50AM +0800, Paul Wise wrote:
> On Tue, 2023-01-17 at 00:55 +0100, Tobias Wackenhut wrote:
> 
> > What would be the cleanest way to package the new version with the 
> > mentioned patch?
> 
> Ask the extension author to get the patch included into the upstream
> rt5 core, get a release of that, then get request-tracker-5 updated.
> 
> Alternatively, drop the patch from the extension upstream and work
> around the issue within the extension upstream codebase instead.
> 
> If neither of these are viable, discuss with the request-tracker-5
> maintainers in Debian about including the patch in that package.

Hi, (somewhat dormant) RT maintainer here.

Thanks Paul, that's spot on. We have in the distant past applied
those sort of patches before they hit RT upstream, but it's definitely
not ideal, and it really depends on the patch. 

However, Tobias, I don't see any patch that needs to be applied in

- Debian already has RT 5.0.3 (>= 5.0.2)?

Let's continue the discussion on
pkg-request-tracker-maintain...@alioth-lists.debian.net (CCed).

Cheers
Dominic

Re: Help patching files in dependency package

[Paul Wise]

On Tue, 2023-01-17 at 00:55 +0100, Tobias Wackenhut wrote:

> What would be the cleanest way to package the new version with the 
> mentioned patch?

Ask the extension author to get the patch included into the upstream
rt5 core, get a release of that, then get request-tracker-5 updated.

Alternatively, drop the patch from the extension upstream and work
around the issue within the extension upstream codebase instead.

If neither of these are viable, discuss with the request-tracker-5
maintainers in Debian about including the patch in that package.

-- 
bye,
pabs

https://wiki.debian.org/PaulWise


signature.asc
Description: This is a digitally signed message part

Help patching files in dependency package

[Tobias Wackenhut]

Hi,

I so far only have basic packaging experience and would be really happy 
for any pointers in the right direction.




Problem (concrete example for simplicity, general topic):

- This is about perl software, i.e. interpreted script files.
- I want to update rt5-extension-resetpassword
- Upstream introduces a patch to rt5 core packaged in request-tracker-5
- Base package is dependency of extension package

What would be the cleanest way to package the new version with the 
mentioned patch? I think I only need pointers on what direction to take 
and maybe a pointer to the docs. I am more interested in the general 
approach to this kind of problem, but the example is the issue at hand.




Ideas so far:

- Package scripts for patch and patch -R
- Including the affected files in the extension package and overriding 
the ones in the dependency
- Somehow integrate the functionality into the base package, that is 
only used by this extension.



Thanks in advance for any hint

Tobias

Re: Help setting dbconfig-common for MariaDB, not MySQL

[Marc Haber]

On Mon, 2 Jan 2023 17:08:25 +0100, Paul Gevers 
wrote:
>Hi Marc,
>
>On 02-01-2023 16:58, Marc Haber wrote:
>> On Mon, 2 Jan 2023 16:31:17 +0100, Paul Gevers 
>> wrote:
>>> On 02-01-2023 14:21, Alessandro Vesely wrote:
 A user complained that MySQL doesn't work, because it misses the INET6
 type that the example settings use.
>>>
>>> And is this an absolute must? (It's an example after all?)
>> 
>> It is. We need to stop having "disable IPv6" as measure 1 if something
>> doesn't work right. It's the default IP protocol for a decade.
>
>Are you saying that MySQL doesn't support IPv6? Or just that the "INET6 
>type" in the context of MariaDB is a MariaDB specific implementation of 
>something? (Sorry, I didn't investigate and assumed the latter).

I didn't investigate and assumed some kind of the former.

Anyway, since we have a diversion between MySQL and MariaDB here that
causes dbconfig-common to trip over an IPv6 issue, I see the usual
solution coming over the horizon and wanted to object against that
one.

Greetings
Marc
-- 
-- !! No courtesy copies, please !! -
Marc Haber |   " Questions are the | Mailadresse im Header
Mannheim, Germany  | Beginning of Wisdom " | 
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 72739834

Re: Help setting dbconfig-common for MariaDB, not MySQL

[Alessandro Vesely]

On Mon 02/Jan/2023 16:31:17 +0100 Paul Gevers wrote:

Hi Alessandro,



Hi, thanks for replying.



On 02-01-2023 14:21, Alessandro Vesely wrote:
please pardon my ignorance about Debian install.  I'm distributing a software 
which could use various DBMS'es by setting a number of parameters.  Example 
parameters are only given for MariaDB.  I distribute a debian/ directory that 
Debian users can use to prepare a package instead of configure, make, make 
install.  However, the debian/postinst supports MariaDB only.


Do I understand you correctly that you don't want to support MySQL?



Yes, it'd be too much work to create, test, and debug the settings for an 
alternative DBMS.



A user complained that MySQL doesn't work, because it misses the INET6 type 
that the example settings use.


And is this an absolute must? (It's an example after all?)



Well the reference example started using INET6 a few years ago, to store both 
IPv4 and IPv6 addresses.  It simplified settings somewhat.  Reverting to the 
previous state is too bad.


A user needing to work with MySQL can replace INET6 with a suitable BLOB, and 
change all the related queries and configs accordingly.  The existing 
debian/postinst won't work in that case.  It could be easily adapted, IF the 
relevant queries and configs were given...



Now I've added "mariadb-client | mariadb-server | dbconfig-no-thanks" to the 
Debian clause in debian/control.


I think that's wrong. At least it would fail to install dbconfig-common in case 
there is a mariadb-client installed. Also, I wonder about the mariadb-server 
part. mariadb-server depends on the versioned mariadb-server-* package which 
depends on the versioned mariadb-client-* package. So in case mariadb-client 
wouldn't be able to be fulfilled, mariadb-server as the second alternative 
isn't going to help. And in my opinion you should not depend on the server 
part. As with most databases, the server part can live on a different host and 
package should really not force the server to be on the same host.



Would "mariadb-client | dbconfig-no-thanks" work?  But see below.


I'm not clear how I could add an (optional) Conflicts mysql-something, also 
because I see no mysql-server in the package cache.


mysql-server is available in unstable, but we don't want to support both MySQL 
and MariaDB in Debian stable at the same time, so currently MySQL is blocked 
from migration. However, derivatives choose differently (Ubuntu supports MySQL 
in their releases).



Indeed, the user who complained was on Ubuntu 22.04 and MySQL version 8.0.23. 
He asked me to add MariaDB to the list of requirements.  Perhaps he can install 
requirements according to what I write in debian/control Depend.  If I only 
require mariadb-client and then the server is MySQL, it won't run.



Is there a way to fail if a user chooses to install the DB but MariaDB is 
missing?  Or is the above enough?


I don't think you can do it with dependencies. If you really want to go this 
route, you have to detect it during run time.



Yeah, not very nice, but still better to discover it at runtime.  The database 
creation with INET6 types will fail on Ubuntu.



Than

Re: Help setting dbconfig-common for MariaDB, not MySQL

[Paul Gevers]

Hi Marc,

On 02-01-2023 16:58, Marc Haber wrote:

On Mon, 2 Jan 2023 16:31:17 +0100, Paul Gevers 
wrote:

On 02-01-2023 14:21, Alessandro Vesely wrote:

A user complained that MySQL doesn't work, because it misses the INET6
type that the example settings use.


And is this an absolute must? (It's an example after all?)


It is. We need to stop having "disable IPv6" as measure 1 if something
doesn't work right. It's the default IP protocol for a decade.


Are you saying that MySQL doesn't support IPv6? Or just that the "INET6 
type" in the context of MariaDB is a MariaDB specific implementation of 
something? (Sorry, I didn't investigate and assumed the latter).


Paul


OpenPGP_signature
Description: OpenPGP digital signature

Re: Help setting dbconfig-common for MariaDB, not MySQL

[Marc Haber]

On Mon, 2 Jan 2023 16:31:17 +0100, Paul Gevers 
wrote:
>On 02-01-2023 14:21, Alessandro Vesely wrote:
>> A user complained that MySQL doesn't work, because it misses the INET6 
>> type that the example settings use.
>
>And is this an absolute must? (It's an example after all?)

It is. We need to stop having "disable IPv6" as measure 1 if something
doesn't work right. It's the default IP protocol for a decade.

Greetings
Marc
-- 
-- !! No courtesy copies, please !! -
Marc Haber |   " Questions are the | Mailadresse im Header
Mannheim, Germany  | Beginning of Wisdom " | 
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 72739834

Re: Help setting dbconfig-common for MariaDB, not MySQL

[Paul Gevers]

Hi Alessandro,

On 02-01-2023 14:21, Alessandro Vesely wrote:
please pardon my ignorance about Debian install.  I'm distributing a 
software which could use various DBMS'es by setting a number of 
parameters.  Example parameters are only given for MariaDB.  I 
distribute a debian/ directory that Debian users can use to prepare a 
package instead of configure, make, make install.  However, the 
debian/postinst supports MariaDB only.


Do I understand you correctly that you don't want to support MySQL? Or 
that you don't know how to support both at the same time? Most packages 
in Debian that are using MariaDB or MySQL can easily support both (hence 
we have the default-mysql-client and virtual-mysql-client packages), and 
indeed dbconfig-common treats them as equal.


A user complained that MySQL doesn't work, because it misses the INET6 
type that the example settings use.


And is this an absolute must? (It's an example after all?)

Now I've added "mariadb-client | 
mariadb-server | dbconfig-no-thanks" to the Debian clause in 
debian/control.


I think that's wrong. At least it would fail to install dbconfig-common 
in case there is a mariadb-client installed. Also, I wonder about the 
mariadb-server part. mariadb-server depends on the versioned 
mariadb-server-* package which depends on the versioned mariadb-client-* 
package. So in case mariadb-client wouldn't be able to be fulfilled, 
mariadb-server as the second alternative isn't going to help. And in my 
opinion you should not depend on the server part. As with most 
databases, the server part can live on a different host and package 
should really not force the server to be on the same host.


I'm not clear how I could add an (optional) Conflicts 
mysql-something, also because I see no mysql-server in the package cache.


mysql-server is available in unstable, but we don't want to support both 
MySQL and MariaDB in Debian stable at the same time, so currently MySQL 
is blocked from migration. However, derivatives choose differently 
(Ubuntu supports MySQL in their releases). As mentioned above, the 
server part can be on a different host, but ependencies are not able to 
describe incompatibility with what runs on the other host.


Is there a way to fail if a user chooses to install the DB but MariaDB 
is missing?  Or is the above enough?


I don't think you can do it with dependencies. If you really want to go 
this route, you have to detect it during run time.


Paul


OpenPGP_signature
Description: OpenPGP digital signature

Help setting dbconfig-common for MariaDB, not MySQL

[Alessandro Vesely]

Hi,

please pardon my ignorance about Debian install.  I'm distributing a software 
which could use various DBMS'es by setting a number of parameters.  Example 
parameters are only given for MariaDB.  I distribute a debian/ directory that 
Debian users can use to prepare a package instead of configure, make, make 
install.  However, the debian/postinst supports MariaDB only.


A user complained that MySQL doesn't work, because it misses the INET6 type 
that the example settings use.  Now I've added "mariadb-client | mariadb-server 
| dbconfig-no-thanks" to the Debian clause in debian/control.  I'm not clear 
how I could add an (optional) Conflicts mysql-something, also because I see no 
mysql-server in the package cache.


Is there a way to fail if a user chooses to install the DB but MariaDB is 
missing?  Or is the above enough?



Thanks in advance for any hint
Ale
--

Re: Help trying to debug an sbuild failure?

[Theodore Ts'o]

On Wed, Dec 28, 2022 at 12:10:51AM +0100, Johannes Schauer Marin Rodrigues 
wrote:
> Note, that if you keep upgrading a Debian unstable chroot across multiple
> releases, it will end up looking slightly different than a freshly
> debootstrapped Debian unstable chroot. So I think there is value in
> semi-regularly re-creating the build chroot from scratch. Maybe write a script
> that does what you need?

That's true, but the number of user/group id's that sbuild would
actually care about is probably quite small and might very well just
be sbuild:sbuild I would think, no?

> Finally, I think this is something that could be solved in sbuild. Ultimately,
> schroot is able to do things as the root user, so it should have sufficient
> permissions to fix up a chroot that carries incorrect permissions. Could you
> quickly note in a bug against sbuild on the Debian BTS which steps exactly you
> carried out so that I am able to reproduce your problem?

Sure, no problem.

> I'm making no promises that I'll find the time to improve the schroot backend
> of sbuild to survive the kind of chroot-rsync that you have performed but if
> this is important to you, then maybe we can make a trade and I implement this
> sbuild functionality and you have a look at pull requests
> https://github.com/tytso/e2fsprogs/pull/118 or #107 and leave some comments in
> return? :)

Thanks for the reminder, I'll take a look.  Most of the patch
proposals for e2fsprogs end up going to linux-e...@vger.kernel.org (so
that other ext4 developers can review them), and I sometimes forgot to
look over the github pull requests.

I'm also about to go on a Panama Canal cruise, where my internet
access may be limited (which is why I was trying to get sbuild setup
on my laptop in the first place :-), and e-mail has the advantage of
being much easily cacheable using offlineimap...

- Ted

Re: Help trying to debug an sbuild failure?

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Theodore Ts'o (2022-12-27 05:19:45)
> On Mon, Dec 26, 2022 at 08:45:53PM +0100, Santiago Vila wrote:
> > El 26/12/22 a las 20:29, Theodore Ts'o escribió:
> > > I: The directory does not exist inside the chroot.
> > 
> > This is really a problem with schroot. I guess that this will not work 
> > either:
> > 
> > schroot -c the-chroot-name
> > 
> > This usually works when you are in your $HOME because this file:
> > 
> > /etc/schroot/default/fstab
> 
> Nope, that's not the issue; yes, /tmp and /home are missing form
> /etc/schroot/sbuild/fstab, but that was true on the *working* setup as
> well, and from what I can tell; that's deliberate.  It looks like the
> goal is to keep things hermetic when building with sbuild, so it's a
> feature that there aren't random directories leaking through from the
> host to the sbuild enviroment.

that is correct. :)

> I think I've figured out the issue.  The problem is that the user and group
> id's for sbuild are different on my desktop and my laptop, and I did an rsync
> to copy the /chroot directories from my desktop to my laptop --- and it
> appears that sbuild is very sensisitve about the user id's being the same
> across the host and chroot environments.
> 
> Apparently sbuild copies the files for the package it is building over
> a directory in /var/lib/sbuild/build, with the permissions being mode
> 770, and that is what sbuild bind mounts into the chroot.  If my
> theory is correct, if the user/group id's are different from between
> the base /etc/passwd and chroot, then things go bad in a hurry.

I think your analysis makes sense.

> >From my working system (while gbp buildpackage was running sbuild):
> 
> % ls -al /var/lib/sbuild/build/
> total 12
> 4 drwxrws--- 3 sbuild sbuild 4096 Dec 26 23:05 ./
> 4 drwxrws--- 4 sbuild sbuild 4096 Dec 19  2020 ../
> 4 drwxrwx--- 3 tytso  sbuild 4096 Dec 26 23:05 f2fs-tools-oT4KHz/
> 
> Amd on the broken (laptop) system:
> 
> # ls -al /var/lib/sbuild/build/
> total 32
> 4 drwxrws--- 8 fwupd-refresh Debian-exim 4096 Dec 26 22:48 ./
> 4 drwxrws--- 4 sbuildsbuild  4096 Dec 25 14:45 ../
> 4 drwxrwx--- 2 tytso Debian-exim 4096 Dec 26 14:01 f2fs-tools-9QfprK/
> 4 drwxrwx--- 2 tytso Debian-exim 4096 Dec 26 16:01 f2fs-tools-btkVPv/
> 4 drwxrwx--- 2 tytso Debian-exim 4096 Dec 26 14:20 f2fs-tools-cVTRAh/
> 4 drwxrwx--- 2 tytso Debian-exim 4096 Dec 26 22:48 f2fs-tools-Myld8j/
> ...
> 
> Each of were created from a failed sbuild invocation...  And
> "Debian-exim" on my laptop has the same group id as "sbuild" on my
> desktop (and which is the group id in my chroots).
> 
> This also explains the error message:
> 
> E: Failed to change to directory ‘/<>’: Permission denied
> 
> Oops.
> 
> So I guess I need to either manually juggle group id's in the chroots
> (and/or my laptop's root directory --- but it's probably easier to do
> it in the chroots, since there are fewer filees to chgrp in the
> chroots), or I could recreate the sbuild chroots from scratch using
> sbuild-createchroot (but then I would need to recreate all of the
> custom hacks so that ccache,eatmydata, apt-auto-proxy, etc. are working
> correctly in the chroot).
> 
> What fun

Note, that if you keep upgrading a Debian unstable chroot across multiple
releases, it will end up looking slightly different than a freshly
debootstrapped Debian unstable chroot. So I think there is value in
semi-regularly re-creating the build chroot from scratch. Maybe write a script
that does what you need?

That being said, I think the biggest problem is, that the main sbuild
contributors these days (me and Jochen) do not use the schroot backend anymore
but the unshare backend instead which doesn't suffer from this kind of problem
(but of course has different quirks that one needs to be aware of). So I must
admit that in the past few years, the schroot backend hasn't gotten the love
that it deserves considering that it's the default backend used on the buildd
machinery.

Finally, I think this is something that could be solved in sbuild. Ultimately,
schroot is able to do things as the root user, so it should have sufficient
permissions to fix up a chroot that carries incorrect permissions. Could you
quickly note in a bug against sbuild on the Debian BTS which steps exactly you
carried out so that I am able to reproduce your problem?

I'm making no promises that I'll find the time to improve the schroot backend
of sbuild to survive the kind of chroot-rsync that you have performed but if
this is important to you, then maybe we can make a trade and I implement this
sbuild functionality and you have a look at pull requests
https://github.com/tytso/e2fsprogs/pull/118 or #107 and leave some comments in
return? :)

Thanks!

cheers, josch

signature.asc
Description: signature

Re: Help trying to debug an sbuild failure?

[Theodore Ts'o]

On Mon, Dec 26, 2022 at 08:45:53PM +0100, Santiago Vila wrote:
> El 26/12/22 a las 20:29, Theodore Ts'o escribió:
> > I: The directory does not exist inside the chroot.
> 
> This is really a problem with schroot. I guess that this will not work either:
> 
> schroot -c the-chroot-name
> 
> This usually works when you are in your $HOME because this file:
> 
> /etc/schroot/default/fstab

Nope, that's not the issue; yes, /tmp and /home are missing form
/etc/schroot/sbuild/fstab, but that was true on the *working* setup as
well, and from what I can tell; that's deliberate.  It looks like the
goal is to keep things hermetic when building with sbuild, so it's a
feature that there aren't random directories leaking through from the
host to the sbuild enviroment.

I think I've figured out the issue.  The problem is that the user and
group id's for sbuild are different on my desktop and my laptop, and I
did an rsync to copy the /chroot directories from my desktop to my
laptop --- and it appears that sbuild is very sensisitve about the
user id's being the same across the host and chroot environments.

Apparently sbuild copies the files for the package it is building over
a directory in /var/lib/sbuild/build, with the permissions being mode
770, and that is what sbuild bind mounts into the chroot.  If my
theory is correct, if the user/group id's are different from between
the base /etc/passwd and chroot, then things go bad in a hurry.

>From my working system (while gbp buildpackage was running sbuild):

% ls -al /var/lib/sbuild/build/
total 12
4 drwxrws--- 3 sbuild sbuild 4096 Dec 26 23:05 ./
4 drwxrws--- 4 sbuild sbuild 4096 Dec 19  2020 ../
4 drwxrwx--- 3 tytso  sbuild 4096 Dec 26 23:05 f2fs-tools-oT4KHz/

Amd on the broken (laptop) system:

# ls -al /var/lib/sbuild/build/
total 32
4 drwxrws--- 8 fwupd-refresh Debian-exim 4096 Dec 26 22:48 ./
4 drwxrws--- 4 sbuildsbuild  4096 Dec 25 14:45 ../
4 drwxrwx--- 2 tytso Debian-exim 4096 Dec 26 14:01 f2fs-tools-9QfprK/
4 drwxrwx--- 2 tytso Debian-exim 4096 Dec 26 16:01 f2fs-tools-btkVPv/
4 drwxrwx--- 2 tytso Debian-exim 4096 Dec 26 14:20 f2fs-tools-cVTRAh/
4 drwxrwx--- 2 tytso Debian-exim 4096 Dec 26 22:48 f2fs-tools-Myld8j/
...

Each of were created from a failed sbuild invocation...  And
"Debian-exim" on my laptop has the same group id as "sbuild" on my
desktop (and which is the group id in my chroots).

This also explains the error message:

E: Failed to change to directory ‘/<>’: Permission denied

Oops.

So I guess I need to either manually juggle group id's in the chroots
(and/or my laptop's root directory --- but it's probably easier to do
it in the chroots, since there are fewer filees to chgrp in the
chroots), or I could recreate the sbuild chroots from scratch using
sbuild-createchroot (but then I would need to recreate all of the
custom hacks so that ccache,eatmydata, apt-auto-proxy, etc. are working
correctly in the chroot).

What fun

- Ted

p.s.  I guess if I had been planning ahead I would have made sure that
various system users and groups which are auto-created by packages at
install-time (and which are therefore different depending on install
order) were pre-created on my laptop with the same numerical id's as
on my desktop, since that would have avoided all *sorts* of random
problems, especially if I was going to play games with copying chroots
around --- or trying to use NFS --- and not getting taken by surprise
by this sort of thing.  Live and learn

Re: Help trying to debug an sbuild failure?

[Santiago Vila]

El 26/12/22 a las 20:29, Theodore Ts'o escribió:

I: The directory does not exist inside the chroot.


This is really a problem with schroot. I guess that this will not work either:

schroot -c the-chroot-name

This usually works when you are in your $HOME because this file:

/etc/schroot/default/fstab

has a line like this:

/home   /home   nonerw,bind 0   0

which makes a bind mount and makes /home to "exist" inside the chroot.

So the solution is to either try sbuild from a directory which is
bind-mounted like that, like $HOME, or add a new entry to fstab
with the directory you need.

Hope this helps.

Help trying to debug an sbuild failure?

[Theodore Ts'o]

Hi, I'm trying to figure out an sbuild failure on my laptop.  The
sbuild environment from replicated from my desktop where things work
perfectly well.  But in my laptop, things are failing at the 
"Setup apt archive" step with

   E: Failed to change to directory ‘/<>’: Permission denied

And I'm completely at a loss trying to figure out what might be going
wrong.  Can anyone give me some hints about what to look for?

Thanks,

- Ted



sbuild (Debian sbuild) 0.84.2 (08 December 2022) on letrec.thunk.org

+==+
| f2fs-tools 1.15.0-1 (amd64)  Mon, 26 Dec 2022 19:20:17 + |
+==+

Package: f2fs-tools
Version: 1.15.0-1
Source Version: 1.15.0-1
Distribution: unstable
Machine Architecture: amd64
Host Architecture: amd64
Build Architecture: amd64
Build Type: full

I: NOTICE: Log filtering will replace 
'var/run/schroot/mount/unstable-amd64-sbuild-a67a3165-6688-4368-a376-66e094e41dfa'
 with '<>'
I: NOTICE: Log filtering will replace 'build/f2fs-tools-cVTRAh/resolver-CwG6Va' 
with '<>'

+--+
| Update chroot|
+--+

Get:1 http://httpredir.debian.org/debian unstable InRelease [161 kB]
Get:2 http://httpredir.debian.org/debian unstable/main Sources.diff/Index [63.6 
kB]
Get:3 http://httpredir.debian.org/debian unstable/main amd64 
Packages.diff/Index [63.6 kB]
Get:4 http://httpredir.debian.org/debian unstable/main Sources 
T-2022-12-26-1404.34-F-2022-12-26-0804.45.pdiff [15.0 kB]
Get:4 http://httpredir.debian.org/debian unstable/main Sources 
T-2022-12-26-1404.34-F-2022-12-26-0804.45.pdiff [15.0 kB]
Get:5 http://httpredir.debian.org/debian unstable/main amd64 Packages 
T-2022-12-26-1404.34-F-2022-12-26-0804.45.pdiff [33.2 kB]
Get:5 http://httpredir.debian.org/debian unstable/main amd64 Packages 
T-2022-12-26-1404.34-F-2022-12-26-0804.45.pdiff [33.2 kB]
Fetched 337 kB in 1s (238 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
Calculating upgrade...
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

+--+
| Fetch source files   |
+--+


Local sources
-

/tmp/gbp/f2fs-tools_1.15.0-1.dsc exists in /tmp/gbp; copying to chroot
I: NOTICE: Log filtering will replace 
'build/f2fs-tools-cVTRAh/f2fs-tools-1.15.0' with '<>'
I: NOTICE: Log filtering will replace 'build/f2fs-tools-cVTRAh' with 
'<>'

+--+
| Install package build dependencies   |
+--+


Setup apt archive
-

Merged Build-Depends: debhelper-compat (= 13), libblkid-dev, libselinux1-dev, 
pkg-config, uuid-dev, build-essential, fakeroot
Filtered Build-Depends: debhelper-compat (= 13), libblkid-dev, libselinux1-dev, 
pkg-config, uuid-dev, build-essential, fakeroot
E: Failed to change to directory ‘/<>’: Permission denied
I: The directory does not exist inside the chroot.  Use the --directory option 
to run the command in a different directory.
Dummy package creation failed
E: Setting up apt archive failed

Setup apt archive
-

Merged Build-Depends: dose-distcheck
Filtered Build-Depends: dose-distcheck
E: Failed to change to directory ‘/<>’: Permission denied
I: The directory does not exist inside the chroot.  Use the --directory option 
to run the command in a different directory.
Dummy package creation failed
E: Setting up apt archive failed
E: Failed to explain bd-uninstallable

+--+
| Summary  |
+--+

Build Architecture: amd64
Build Type: full
Build-Space: n/a
Build-Time: 0
Distribution: unstable
Fail-Stage: explain-bd-uninstallable
Host Architecture: amd64
Install-Time: 0
Job: /tmp/gbp/f2fs-tools_1.15.0-1.dsc
Machine Architecture: amd64
Package: f2fs-tools
Package-Time: 0
Source-Version: 1.15.0-1
Space: n/a
Status: given-back
Version: 1.15.0-1

Finished at 2022-12-26T19:20:17Z
Build needed 00:00:00, no disk space

Re: Help needed for watch file after bitbucket changed download page

[Juri Grabowski]

Hello Andreas,

On 2022-09-29 14:54 4, Andreas Tille wrote:

I confirm this works.  However, uscan does not do the usual link to
orig.tar.gz.  Any idea why this is the case?

I have found this bug:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=896705

But did you have signing key from Rob Egan , because I
didn't found it on keyserver or in debian/upstream/signing-key.asc

Best Regards,
Juri Grabowski

Bitbucket changed download page - possibly lots of watch files affected (Was: Help needed for watch file after bitbucket changed download page)

[Andreas Tille]

Hi Nilesh,

Am Thu, Sep 29, 2022 at 03:43:58PM +0530 schrieb Nilesh Patra:
> I don't know how to go about it, but I suspect -devel might be a better list 
> for this since this is a general question and many package maintainers could 
> be using bitbucket sources.

While I considered debian-mentors@l.d.o usually the proper list to find
a solution for problems in watch files I agree that this should be made
public also on debian-devel.  It seems quite some packages are affected:

   
https://codesearch.debian.net/search?q=path%3Adebian%2Fwatch+bitbucket&literal=1

My guess is that most of these watch files might be not working any more.
May be gitmode as proposed by Juri Grabowski[2] is a promising solution
(but as I wrote in my response it does not create the link to orig.tar.gz).

Kind regards

 Andreas.
 
> On 29 September 2022 12:54:47 pm IST, Andreas Tille  wrote:
> >Hi,
> >
> >the watch file for metabat[1] used to work nicely until some point in
> >time when bitbucket replaced `v@ANY_VERSION@` by the commit ID which is
> >not sensibly sorting any more.  Is there any trick how I can get
> >bitbucket pages working again?
> >
> >Kind regards
> >
> > Andreas.
> >
> >[1] https://salsa.debian.org/med-team/metabat/-/blob/master/debian/watch

[2] https://lists.debian.org/debian-mentors/2022/09/msg00153.html 

-- 
http://fam-tille.de

help with chromium

[Andres Salomon]

Hi,

Would you like to see chromium in the bookworm release? The release 
managers really want to see a team maintaining it. I filed a RFH bug, 
there's plenty to do: https://bugs.debian.org/1016047


Please consider joining! No need to spam the list, you can just email 
me privately if you're interested in helping out.


Thanks,
Andres

Re: Help needed with a dh_shlibs failure on non amd64 platforms

[Simon McVittie]

On Mon, 04 Jul 2022 at 09:29:54 +0200, Martin Quinson wrote:
> All libraries (eg libns3-bridge.so.36.1) that it cannot find are part of the
> package. They are added to the debian/libns3.36/DEBIAN/shlibs

Independent of the dh_shlibdeps failure that was already diagnosed, if
these libraries are public (i.e. other packages in Debian are allowed to
link against them) and the SONAMEs of the libraries are of the form
libns3-*.so.36.1, then the binary package that contains them should
probably be called something like libns3-36.1, rather than libns3.36.

(Either that, or they should be split up into one binary package per
library, named like libns3-bridge36.1 and so on, by mechanically
transforming their SONAMEs according to the convention documented
in Policy - but I can understand why you would prefer not to do that when
there are a large number of libraries with matching versioning.)

Otherwise, when the SONAMEs jump from libns3-*.so.36.1 to libns3-*.so.36.2
in a future version, any dependent packages that require libns3-*.so.36.1
would be broken by upgrading libns3.36 to a version that no longer contains
libns3-*.so.36.1.

> If you wonder, the cmake macro to define and build a library is in 
>   ns-3.36.1/build-support/custom-modules/ns3-module-macros.cmake
> I already had to patch it to support Debian:
> https://salsa.debian.org/debian/ns3/-/blob/master/debian/patches/library-soversion.diff
> This patch is ugly for now, but I'm already discussing with upstream so that
> they integrate the spirit of this patch to their code. They are receptive.

If you're giving advice to upstream, it's important to be aware of
whether the libraries are private (only to be used by other binary
packages within the same source package, like samba-libs or
libsystemd-shared) or public (with a -dev package that can validly be
used by other source packages, like libsmbclient or libsystemd0 or any
typical shared library like GTK or Qt). The correct advice to give to an
upstream for private shared libraries is not the same as the correct
advice for public shared libraries.

smcv

Re: Help needed with a dh_shlibs failure on non amd64 platforms

[Samuel Thibault]

Hello,

Martin Quinson, le lun. 04 juil. 2022 09:29:54 +0200, a ecrit:
>   dpkg-shlibdeps -Tdebian/libns3.36.substvars 
> debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-wimax.so.36.1

x86_64-linux-gnu is only valid for amd64.

In

./debian/rules: -DCMAKE_INSTALL_RUNSTATEDIR=/run 
-DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_INSTALL_LIBDIR=lib/x86_64-linux-gnu 
-DCMAKE_INSTALL_PREFIX=/usr

you want to use $(DEB_HOST_MULTIARCH) instead of hardcoding x86_64-linux-gnu

Samuel

Help needed with a dh_shlibs failure on non amd64 platforms

[Martin Quinson]

Hello all,

I come to you because I'm puzzled with a bug I have in one of my package, and
I'm seeking for help. Please CC me when answering as I'm not on this list.

The package is ns3, a scientific simulator of computer networks. This package is
huge, I seem to be the only active maintainer, but upstream is very
collaborative.

Upstream just moved from a build system called waf to cmake, which is an nice
move. They introduced a small python script saving the waf interface to their
users that don't like changes, and unfortunately the raw cmake interface is not
usable yet (cmake checks on files created by the script), so I cannot use the
plain classical cmake build in debian/rules. I did my best to mimick the
behavior of `dh --buildsystem=cmake` but I have a strange failure on non-amd64
platforms:
https://buildd.debian.org/status/package.php?p=ns3

The build, tests and install targets go well, until dh_shlibdeps. At that step,
I get a huge bunch of errors like the following:
```
   dh_shlibdeps -a
dpkg-shlibdeps -Tdebian/libns3.36.substvars 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-wimax.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-wifi.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-wave.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-visualizer.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-virtual-net-device.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-uan.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-traffic-control.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-topology-read.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-tap-bridge.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-stats.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-spectrum.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-sixlowpan.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-propagation.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-point-to-point.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-point-to-point-layout.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-olsr.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-nix-vector-routing.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-network.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-netanim.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-mobility.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-mesh.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-lte.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-lr-wpan.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-internet.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-internet-apps.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-flow-monitor.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-fd-net-device.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-energy.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-dsr.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-dsdv.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-csma.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-csma-layout.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-core.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-config-store.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-buildings.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-bridge.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-applications.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-aodv.so.36.1 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-antenna.so.36.1
dpkg-shlibdeps: error: cannot find library libns3-bridge.so.36.1 needed by 
debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-internet.so.36.1 (ELF format: 
'elf64-littleaarch64' abi: '020100b7'; RPATH: '')
dpkg-shlibdeps: error: cannot find library libns3-traffic-control.so.36.1 
needed by debian/libns3.36/usr/lib/x86_64-linux-gnu/libns3-internet.so.36.1 
(ELF format: 'elf64-littleaarch64' abi: '020100b7'; RPATH: '')
```
This specific one is for arm64, but I get exactly the same problem for all
platforms but amd64.
https://buildd.debian.org/status/fetch.php?pkg=ns3&arch=arm64&ver=3.36.1%2Bdfsg-2&stamp=1656893780&raw=0

All libraries (eg libns3-bridge.so.36.1) that it cannot find are part of the
package. They are added to the debian/libns3.36/DEBIAN/shlibs a few lines above
in the build log, and dh_strip found them further above in the log. What really
puzzles me is that the package builds fine on amd64 and i386. 

The package is uptodate on salsa, in case someone wants to test something
directly on the package. Fear not to do so, this package only takes one hour to
build on my machine :)

If you wonder, the cmake macro to define and build a library is in 
  ns-

Help in setting up lxc for autopkgtest - autopkgtest-build-lxc fails

[Julian Gilbey]

Following Simon's suggestion, I decided to try setting up lxc to use
autopkgtest-lxc, mimicking the ci.debian.org setup.  I haven't managed
to do so yet, and have run into lots of problems.  I'd really
appreciate some advice on what to try, and them we can record advice
somewhere on the Debian wiki.

Following the advice in autopkgtest-build-lxc(1), that user containers
will not work with many or even most autopkgtests, I ran it as route.

Step 1: Install the lxc and autopkgtest packages
That went smoothly.  (lxc version 1:4.0.11-1, autopkgtest version 5.21)

Step 2: Run the command "autopkgtest-build-lxc debian sid"
I got various warning messages, and this essentially failed...

lxc-create: autopkgtest-sid: storage/btrfs.c: btrfs_create: 938 Inappropriate 
ioctl for device - Failed to create btrfs subvolume 
"/var/lib/lxc/autopkgtest-sid/rootfs"
lxc-create: autopkgtest-sid: storage/zfs.c: zfs_create: 735 Failed to create 
zfs dataset "zfs:lxc/autopkgtest-sid": lxc-create: autopkgtest-sid: utils.c: 
run_command_internal: 1588
lxc-create: autopkgtest-sid: storage/lvm.c: do_lvm_create: 165 Failed to create 
logical volume "autopkgtest-sid":   Volume group "lxc" not found
  Cannot process volume group lxc
lxc-create: autopkgtest-sid: storage/lvm.c: lvm_create: 623 Error creating new 
logical volume "lvm:/dev/lxc/autopkgtest-sid" of size "1073741824 bytes"
debootstrap is /usr/sbin/debootstrap
Checking cache download in /var/cache/lxc/debian/rootfs-sid-amd64 ... 
Downloading debian minimal ...
I: Target architecture can be executed
I: Retrieving InRelease 
[downloading and installing base system]
I: Base system installed successfully.
Download complete.
Copying rootfs to /var/lib/lxc/autopkgtest-sid/rootfs...ERROR: ld.so: object 
'libeatmydata.so' from LD_PRELOAD cannot be preloaded (cannot open shared 
object file): ignored.
ERROR: ld.so: object 'libeatmydata.so' from LD_PRELOAD cannot be preloaded 
(cannot open shared object file): ignored.
Generating locales (this might take a while)...
ERROR: ld.so: object 'libeatmydata.so' from LD_PRELOAD cannot be preloaded 
(cannot open shared object file): ignored.
  en_GB.UTF-8ERROR: ld.so: object 'libeatmydata.so' from LD_PRELOAD cannot be 
preloaded (cannot open shared object file): ignored.
[... lots more libeatmydata.so warnings, interspersed with other
information messages ...]
ERROR: ld.so: object 'libeatmydata.so' from LD_PRELOAD cannot be preloaded 
(cannot open shared object file): ignored.
lxc-start: autopkgtest-sid: lxccontainer.c: wait_on_daemonized_start: 867 
Received container state "ABORTING" instead of "RUNNING"
lxc-start: autopkgtest-sid: tools/lxc_start.c: main: 306 The container failed 
to start
lxc-start: autopkgtest-sid: tools/lxc_start.c: main: 309 To get more details, 
run the container in foreground mode
lxc-start: autopkgtest-sid: tools/lxc_start.c: main: 311 Additional information 
can be obtained by setting the --logfile and --logpriority options

Something weird is going on here, but autopkgtest-build-lxc doesn't
seem to allow a --logfile option.

I attempted to start it manually, using the command
  lxc-start -n autopkgtest-sid --logfile /tmp/lxc.log --logpriority INFO
and got the following errors in the log file:

lxc-start autopkgtest-sid 20220429124743.756 WARN cgfsng - 
cgroups/cgfsng.c:get_hierarchy:142 - There is no useable devices controller
lxc-start autopkgtest-sid 20220429124743.756 ERRORcgfsng - 
cgroups/cgfsng.c:cg_legacy_set_data:2675 - No such file or directory - Failed 
to setup limits for the "devices" controller. The controller seems to be unused 
by "cgfsng" cgroup driver or not enabled on the cgroup hierarchy
lxc-start autopkgtest-sid 20220429124743.756 ERRORcgfsng - 
cgroups/cgfsng.c:cgfsng_setup_limits_legacy:2742 - No such file or directory - 
Failed to set "devices.deny" to "a"
lxc-start autopkgtest-sid 20220429124743.756 ERRORstart - 
start.c:lxc_spawn:1890 - Failed to setup legacy device cgroup controller limits
lxc-start autopkgtest-sid 20220429124743.756 ERRORlxccontainer - 
lxccontainer.c:wait_on_daemonized_start:867 - Received container state 
"ABORTING" instead of "RUNNING"
[...]


I found something like this reported at this GitHub issue against lxc:
https://github.com/lxc/lxc/issues/2268
so I followed the advice there and ran the commands:

mount -o remount,rw /sys/fs/cgroup
mkdir /sys/fs/cgroup/devices
mount -t cgroup devices -o devices /sys/fs/cgroup/devices
mount -o remount,ro /sys/fs/cgroup

But that seems to be really bad, as now systemd-logind.service seems
to have broken and cannot be restarted, so I don't recommend doing
that!

I've restarted my system and started again.  The above solution is
very bad at least partly because /sys/fs/cgroup is type cgroup2.  But
I still can't start the LXC container, which makes running autopkgtest
impossible.

Is this a bug in lxc or in autopkgtest-build-lxc or somewhere else?

Any suggestions would be much appreciated!

   Julian

Bug#1009331: ITP: node-gitlab-favicon-overlay -- Combine images for a favicon with the help of canvas

[Michael Ikwuegbu]

Package: wnpp
Severity: wishlist
Owner: Michael Ikwuegbu 
X-Debbugs-CC: debian-devel@lists.debian.org

* Package name: node-gitlab-favicon-overlay
  Version : 2.0.0
  Upstream Author  : GitLab Frontend Team 
* URL  :
https://gitlab.com/gitlab-org/frontend/favicon-overlay#read>
* License : Expat
  Programming Lang: JavaScript
  Description : Combine images for a favicon with the help of
canvas
.
 Node.js is an event-based server-side JavaScript engine.

Bug#1007107: ITP: pytz-deprecation-shim -- Shims to help you safely remove pytz

[Edward Betts]

Package: wnpp
Severity: wishlist
Owner: Edward Betts 
X-Debbugs-Cc: debian-devel@lists.debian.org, debian-pyt...@lists.debian.org

* Package name: pytz-deprecation-shim
  Version : 0.1.0.post0
  Upstream Author : Paul Ganssle 
* URL : https://github.com/pganssle/pytz-deprecation-shim
* License : Apache 2.0
  Programming Lang: Python
  Description : Shims to help you safely remove pytz

pytz has served the Python community well for many years, but it is no
longer the best option for providing time zones. pytz has a non-standard
interface that is very easy to misuse; this interface was necessary when
pytz was created, because datetime had no way to represent ambiguous
datetimes, but this was solved in in Python 3.6, which added a fold
attribute to datetimes in PEP 495. With the addition of the zoneinfo
module in Python 3.9 (PEP 615), there has never been a better time to
migrate away from pytz.

However, since pytz time zones are used very differently from a standard
tzinfo, and many libraries have built pytz zones into their standard
time zone interface (and thus may have users relying on the existence of
the localize and normalize methods); this library provides shim classes
that are compatible with both PEP 495 and pytz's interface, to make it
easier for libraries to deprecate pytz.

This module is required by the latest version of python-tzlocal.

I plan to maintain this package as part of the Python modules team.

Re: DD(s) to help DM land some long-overdue package updates?

[Pierre-Elliott Bécue]

Le 26 février 2022 16:28:18 GMT+01:00, Reuben Thomas  a écrit :
>On Fri, 4 Feb 2022 at 14:30, Reuben Thomas  wrote:
>
>>
>> Having just now got the new Debian packaging building without error, I
>> shall now follow the ITS procedure and see what happens!
>>
>
>I have waited 8 days since posting debdiffs, and had no response, so I've
>now filed an ITS bug: #1006481.
>
>-- 
>https://rrt.sc3d.org

ACK ! 
--
Pierre-Elliott Bécue
From my phone

Re: DD(s) to help DM land some long-overdue package updates?

[Reuben Thomas]

On Fri, 4 Feb 2022 at 14:30, Reuben Thomas  wrote:

>
> Having just now got the new Debian packaging building without error, I
> shall now follow the ITS procedure and see what happens!
>

I have waited 8 days since posting debdiffs, and had no response, so I've
now filed an ITS bug: #1006481.

-- 
https://rrt.sc3d.org

Re: DD(s) to help DM land some long-overdue package updates?

[Reuben Thomas]

On Sat, 1 Jan 2022 at 22:34, Reuben Thomas  wrote:

> On Sat, 1 Jan 2022 at 20:43, Pierre-Elliott Bécue  wrote:
> >
> > I suggest this path:
> >
> > Send bug reports with your debdiff proposals for each package. If 8 days
> > after you get no reply from the maintainer, file an ITS against the
> > packages for which you got no reply.
> >
> > If at the end of the process, the ITS pass, I'll make your uploads
> > after reviewing the changes.
>
> Many thanks, I'll try to do this.
>

I thought I should give an update on my progress: Santiago Vila started
work on packaging recode 3.7, one of the packages I mentioned, so I did
some work to help with that. That has actually taken most of my time until
now.

However, I have also found some time to work on the packaging of mmv. I
think this is one of the clearest-cut packages in the list I mentioned: I
am the (new) upstream of mmv, and I have made a new release.

Having just now got the new Debian packaging building without error, I
shall now follow the ITS procedure and see what happens!

-- 
https://rrt.sc3d.org

Re: Help needed: C++ compile error with nix 2.5.1

[Jakub Wilk]

* Thomas Koch , 2022-01-16, 11:23:

https://github.com/NixOS/nix/issues/5923


The first bad commit is:
https://github.com/nlohmann/json/commit/0e694b4060ed55df

(This happens to be also the first bad commit for 
, so I guess it's the same 
issue.)


I don't know what's going on here, but the attached patch seems to fix 
it.


--
Jakub Wilk
diff --git a/src/libstore/realisation.cc b/src/libstore/realisation.cc
index f871e6437..eb3ca0198 100644
--- a/src/libstore/realisation.cc
+++ b/src/libstore/realisation.cc
@@ -78,7 +78,7 @@ Realisation Realisation::fromJSON(
 auto fieldIterator = json.find(fieldName);
 if (fieldIterator == json.end())
 return std::nullopt;
-return *fieldIterator;
+return std::string(*fieldIterator);
 };
 auto getField = [&](std::string fieldName) -> std::string {
 if (auto field = getOptionalField(fieldName))

Help needed: C++ compile error with nix 2.5.1

[Thomas Koch]

For a C++ programmer who cares about nix in Debian:
https://github.com/NixOS/nix/issues/5923

Thank you! Thomas

Re: DD(s) to help DM land some long-overdue package updates?

[Reuben Thomas]

On Sat, 1 Jan 2022 at 20:55, Wookey  wrote:
>
> Hi Rebuen. I helped you with the last libpaper refurbishment in
> 2012-2014. Happy to do that again, although as people have pointed out
> complete rewrites are not really NMU material and we should follow the
> salvage process.
>
> Well done for getting those boring old packages into better shape (again).

Many thanks! I'll follow up PEB's offer, and see what happens.

-- 
https://rrt.sc3d.org

Re: DD(s) to help DM land some long-overdue package updates?

[Reuben Thomas]

On Sat, 1 Jan 2022 at 20:43, Pierre-Elliott Bécue  wrote:
>
> I suggest this path:
>
> Send bug reports with your debdiff proposals for each package. If 8 days
> after you get no reply from the maintainer, file an ITS against the
> packages for which you got no reply.
>
> If at the end of the process, the ITS pass, I'll make your uploads
> after reviewing the changes.

Many thanks, I'll try to do this.

-- 
https://rrt.sc3d.org

Re: DD(s) to help DM land some long-overdue package updates?

[Holger Levsen]

On Sat, Jan 01, 2022 at 02:48:49PM -0500, Boyuan Yang wrote:
> should be the correct choice. For a specific example, I just spotted
> https://bugs.debian.org/961136 the second time (last time back in May 2020);

yet the bug was never pinged after it was filed. and so as many bugs it
"got lost", despite being there. I wouldn't blame a busy maintainer for that.
I'd rather blame myself for not pinging the bug in >18 month despite caring.

it's ok to ping bugs without an answer (after a sensible amount of time), and
it's ok to ping again and again (again, after sensible time).


-- 
cheers,
Holger

 ⢀⣴⠾⠻⢶⣦⠀
 ⣾⠁⢠⠒⠀⣿⡁  holger@(debian|reproducible-builds|layer-acht).org
 ⢿⡄⠘⠷⠚⠋⠀  OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
 ⠈⠳⣄

Stop saying that we are all in the same boat.
We’re all in the same storm. But we’re not all in the same boat.


signature.asc
Description: PGP signature

Re: DD(s) to help DM land some long-overdue package updates?

[Wookey]

On 2022-01-01 19:06 +, Reuben Thomas wrote:
> Here's a summary of the packages I'm trying to update. If any DD can help me
> get these uploaded, I'd be most grateful,

Hi Rebuen. I helped you with the last libpaper refurbishment in
2012-2014. Happy to do that again, although as people have pointed out
complete rewrites are not really NMU material and we should follow the
salvage process.

Well done for getting those boring old packages into better shape (again). 

Wookey
-- 
Principal hats:  Linaro, Debian, Wookware, ARM
http://wookware.org/


signature.asc
Description: PGP signature

Re: DD(s) to help DM land some long-overdue package updates?

[Pierre-Elliott Bécue]

Reuben Thomas  wrote on 01/01/2022 at 20:52:15+0100:

> On Sat, 1 Jan 2022 at 19:48, Boyuan Yang  wrote:
>>
>> That being said, while providing a list onto debian-devel is not a bad idea,
>> https://www.debian.org/doc/manuals/developers-reference/pkgs.en.html#package-salvaging
>> should be the correct choice. For a specific example, I just spotted
>> https://bugs.debian.org/961136 the second time (last time back in May 2020);
>> let me know if you would like to initiate a package salvaging process on this
>> certain package and I can help.
>
> Thanks very much for the pointer! I was not aware of this process. It
> might be appropriate for me, I guess, but I'd still need DD help.

I suggest this path:

Send bug reports with your debdiff proposals for each package. If 8 days
after you get no reply from the maintainer, file an ITS against the
packages for which you got no reply.

If at the end of the process, the ITS pass, I'll make your uploads
after reviewing the changes.

Cheers,
-- 
PEB
Both as a DD and as a MIA team member.

Re: DD(s) to help DM land some long-overdue package updates?

[Reuben Thomas]

On Sat, 1 Jan 2022 at 19:48, Boyuan Yang  wrote:
>
> That being said, while providing a list onto debian-devel is not a bad idea,
> https://www.debian.org/doc/manuals/developers-reference/pkgs.en.html#package-salvaging
> should be the correct choice. For a specific example, I just spotted
> https://bugs.debian.org/961136 the second time (last time back in May 2020);
> let me know if you would like to initiate a package salvaging process on this
> certain package and I can help.

Thanks very much for the pointer! I was not aware of this process. It
might be appropriate for me, I guess, but I'd still need DD help.

-- 
https://rrt.sc3d.org

Re: DD(s) to help DM land some long-overdue package updates?

[Boyuan Yang]

在 2022-01-02星期日的 00:33 +0500，Andrey Rahmatullin写道：
> On Sat, Jan 01, 2022 at 07:06:09PM +, Reuben Thomas wrote:
> > Here's a summary of the packages I'm trying to update. If any DD can help
> > me
> > get these uploaded, I'd be most grateful
> The packages you listed are not orphaned and the changes you listed don't
> warrant NMUs so your only options are either sending your updated packages
> to their current maintainers, or
> https://www.debian.org/doc/manuals/developers-reference/pkgs.en.html#package-salvaging
> 
> > I really just need a sponsor for uploads 
> Well, that's not how it works for non-orphaned packages.
> Otherwise, see https://mentors.debian.net/sponsors/

I would suggest DDs with spare time to take a deeper look into the package
list provided by Reuben. Personally I found many of them neglected by their
package maintainers.

That being said, while providing a list onto debian-devel is not a bad idea,
https://www.debian.org/doc/manuals/developers-reference/pkgs.en.html#package-salvaging
should be the correct choice. For a specific example, I just spotted
https://bugs.debian.org/961136 the second time (last time back in May 2020);
let me know if you would like to initiate a package salvaging process on this
certain package and I can help.

Thanks,
Boyuan Yang


signature.asc
Description: This is a digitally signed message part

Re: DD(s) to help DM land some long-overdue package updates?

[Andrey Rahmatullin]

On Sat, Jan 01, 2022 at 07:06:09PM +, Reuben Thomas wrote:
> Here's a summary of the packages I'm trying to update. If any DD can help me
> get these uploaded, I'd be most grateful
The packages you listed are not orphaned and the changes you listed don't
warrant NMUs so your only options are either sending your updated packages
to their current maintainers, or
https://www.debian.org/doc/manuals/developers-reference/pkgs.en.html#package-salvaging

> I really just need a sponsor for uploads 
Well, that's not how it works for non-orphaned packages.
Otherwise, see https://mentors.debian.net/sponsors/

-- 
WBR, wRAR


signature.asc
Description: PGP signature

DD(s) to help DM land some long-overdue package updates?

[Reuben Thomas]

I'm a long-term Debian user and upstream maintainer, and a DM.

I'm working to package updates to mature packages. All of them have
maintainers who are at best sporadically responsive—I do not blame them! I'm
sure they're all hugely overworked. (I'm also upstream for several packages
with responsive maintainers who are all excellent at packaging new
releases.) I've contacted various DDs who have helped me in the past, but
over the last few years they've all been unresponsive too—again, I'm sure
they are all swamped with higher-priority things to do.

In particular, if you're the maintainer of one of the packages I mention,
and you're reading this, please don't take this appeal as any sort of
criticism! If you've not had a recent communication for me, it may just be a
spam trap problem—do get in touch if you can!

Here's a summary of the packages I'm trying to update. If any DD can help me
get these uploaded, I'd be most grateful, it should also be good for users,
as the upstreams I maintain contain in some cases many years of fixes and
improvements relative to the current Debian version. In all cases, I'm happy
to do the Debian packaging work too, and I have up-to-date packaging for
most of the packages.

* recode: recode 3.7 contains many years of fixes and improvements over the
  packaged 3.6, since the death of recode's author, François Pinard. The
  current release fixes almost all the outstanding issues in the BTS.

* libpaper: I have completely rewritten libpaper, while keeping it
  compatible with the current version. It is more friendly to users, who can
  now have their own custom paper sizes, and to programs written in
  languages other than C, which can use it via the new "paper" command.

* psutils: I have rewritten psutils in Perl (was in C), fixing bugs, and
  moving most of the functionality into pstops(1); most of the commands are
  now implemented as invocations of pstops.

* finddup: this package used to be called perforate; it was removed from
  testing on 27th December. I renamed it (the old name confusing, as there
  was no "perforate" command, and these days the most useful functionality
  is the “finddup” command), fixed bugs, and simplified the code.

* mmv: I have added the ability for mmv to rename directories, fixed bugs,
  and simplified the code.

I really just need a sponsor for uploads and any other parts of the process
(e.g. review of "finddup" as it's technically a new package) that I can't do
myself as a mere DM.

Feel free either to reply to this message or drop me a line directly; thanks
in advance to anyone who's able to help!

-- 
https://rrt.sc3d.org

Bug#1000809: ITP: qstylizer -- Python package to help with the construction of PyQt/PySide stylesheets

[Julian Gilbey]

Package: wnpp
Severity: wishlist
Owner: Julian Gilbey 
X-Debbugs-Cc: debian-devel@lists.debian.org

* Package name: qstylizer
  Version : 0.2.1
  Upstream Author : Brett Lambright
* URL : https://github.com/blambright/qstylizer
* License : MIT
  Programming Lang: Python
  Description : Python package to help with the construction of PyQt/PySide 
stylesheets

This package allows for both the construction of new stylesheets and
the easy modification of existing ones.


This package is a dependency of the new version of Spyder
(python3-spyder).

It will be maintained within the Debian Python Team.

Re: Help porting Ceph 16.2.6 to mips6el, mipsel and armel

[Thomas Goirand]

Hi Simon,

Thanks a lot for your help.

On 11/20/21 12:55 PM, Simon McVittie wrote:
> [...]
> You'll notice -latomic appears *before* the various .a and .o files, but
> in general, link order matters: each object on the linker command-line is
> only used to satisfy the dependencies of objects that appear before it.
> So you'll want -latomic to appear later, next to other dependencies
> like -ldl.

This was completely right, though the reason for it seems to be the
cmake test failure to take mips64el into account. Hopefully, this patch
fixes it:

https://salsa.debian.org/ceph-team/ceph/-/blob/debian/unstable/debian/patches/cmake-test-for-16-bytes-atomic-support-on-mips-also.patch

> On armel, the error seems to be:
> 
>> /tmp/cc3CvITT.s:2594: Error: selected processor does not support `yield' in 
>> ARM mode
>> make[7]: *** [CMakeFiles/rocksdb.dir/build.make:3436: 
>> CMakeFiles/rocksdb.dir/third-party/folly/folly/synchronization/DistributedMutex.cpp.o]
>>  Error 1
> 
> so probably there is some inline assembly that assumes ARMv6 or later and
> does not account for armel's ARMv5-based baseline.

Hopefully, this patch fixes it:
https://salsa.debian.org/ceph-team/ceph/-/blob/debian/unstable/debian/patches/only-yied-under-armv7-and-above.patch

> On mipsel, it looks like 32 bits of address space might not be enough,
> so you might need to try the same tricks that e.g. webkit2gtk uses to
> save address space:
> 
>> virtual memory exhausted: Cannot allocate memory
>> make[3]: *** [src/msg/CMakeFiles/common-msg-objs.dir/build.make:93: 
>> src/msg/CMakeFiles/common-msg-objs.dir/Message.cc.o] Error 1

I've added a few tricks to hopefully save on memory at build time on 32
bits arch, I'm not sure this will be enough though.

Anyways, now I have a different failure, on all arch this time:

/usr/include/boost/container/detail/copy_move_algo.hpp:1083:10: error:
‘__fallthrough__’ was not declared in this scope; did you mean
‘fallthrough’?
 1083 |  BOOST_FALLTHROUGH;
  |  ^

Any idea what can cause this? To me, this looks like related to the
latest gcc update (but not sure)...

Cheers,

Thomas Goirand (zigo)

Re: Help porting Ceph 16.2.6 to mips6el, mipsel and armel

[Simon McVittie]

On Sat, 20 Nov 2021 at 11:31:04 +0100, Thomas Goirand wrote:
> Latest Ceph doesn't build on 3 arch:
> 
> https://buildd.debian.org/status/package.php?p=ceph&suite=experimental
> 
> (plus the unofficial ports...)
> 
> What worries me the most is mips6el, where the linker says "undefined
> reference to `__atomic_load_16'" (and more like this). I don't
> understand because there really is a -latomic parameter to GCC when
> linking, so it should be working.

If I download and unpack libatomic1_11.2.0-10_mips64el.deb,
libatomic1_11.2.0-10_mips64el/DEBIAN/symbols mentions symbol
__atomic_load_16@LIBATOMIC_1.0, so -latomic should have been
sufficient.

I think this is a problem with compiler argument order. It looks as though
the failing link might be this one (one long line in the log, newlines
added here to make it more comprehensible):

/usr/bin/c++ \
-g \
-O2 \
-ffile-prefix-map=/<>=. \
-fstack-protector-strong \
-Wformat \
-Werror=format-security \
-Wdate-time \
-D_FORTIFY_SOURCE=2 \
-O2 \
-g \
-DNDEBUG \
-Wl,-z,relro \
-Wl,-z,now \
-Wl,--as-needed \
-latomic \
-rdynamic \
-pie \
CMakeFiles/rbd-mirror.dir/main.cc.o \
-o \
../../../bin/rbd-mirror \
 \
-Wl,-rpath,/<>/obj-mips64el-linux-gnuabi64/lib: \
../../../lib/librbd_mirror_internal.a \
../../../lib/librbd_mirror_types.a \
../../../lib/librbd_api.a \
../../../lib/librbd_internal.a \
../../../lib/librbd_types.a \
../../../lib/libjournal.a \
../../../lib/liblibneorados.a \
../../../lib/librados.so.2.0.0 \
../../../lib/libosdc.a \
../../../lib/libcls_rbd_client.a \
../../../lib/libcls_lock_client.a \
../../../lib/libcls_journal_client.a \
../../../lib/libglobal.a \
../../../lib/libheap_profiler.a \
/usr/lib/mips64el-linux-gnuabi64/libtcmalloc.so \
/usr/lib/mips64el-linux-gnuabi64/libssl.so \
/lib/mips64el-linux-gnuabi64/libcryptsetup.so \
../../../lib/libceph-common.so.2 \
../../../lib/libfmt.a \
/usr/lib/mips64el-linux-gnuabi64/libblkid.so \
/usr/lib/mips64el-linux-gnuabi64/libcrypto.so \
../../../lib/libjson_spirit.a \
../../../lib/libcommon_utf8.a \
../../../lib/liberasure_code.a \
../../../lib/libcrc32.a \
../../../lib/libarch.a \
/usr/lib/mips64el-linux-gnuabi64/libboost_thread.so.1.74.0 \
/usr/lib/mips64el-linux-gnuabi64/libboost_atomic.so.1.74.0 \
/usr/lib/mips64el-linux-gnuabi64/libboost_random.so.1.74.0 \
/usr/lib/mips64el-linux-gnuabi64/libboost_system.so.1.74.0 \
/usr/lib/mips64el-linux-gnuabi64/libboost_program_options.so.1.74.0 \
/usr/lib/mips64el-linux-gnuabi64/libboost_date_time.so.1.74.0 \
/usr/lib/mips64el-linux-gnuabi64/libboost_iostreams.so.1.74.0 \
-lstdc++fs \
-pthread \
/usr/lib/mips64el-linux-gnuabi64/libudev.so \
/usr/lib/mips64el-linux-gnuabi64/libibverbs.so \
/usr/lib/mips64el-linux-gnuabi64/librdmacm.so \
-ldl \
/usr/lib/mips64el-linux-gnuabi64/librt.so \
-lresolv

You'll notice -latomic appears *before* the various .a and .o files, but
in general, link order matters: each object on the linker command-line is
only used to satisfy the dependencies of objects that appear before it.
So you'll want -latomic to appear later, next to other dependencies
like -ldl.

A common reason for this to be a problem is adding -latomic (or some
other library) to Automake's foo_LDFLAGS instead of the correct foo_LIBS
or foo_LIBADD, or the equivalent in other build systems. I see ceph is
built with CMake: I don't know the specifics of how this fits together in
CMake, but most likely it has the same distinction between dependencies
and non-dependency linker options that e.g. Automake and Meson do.

Regarding the other failing release architectures:

On armel, the error seems to be:

> /tmp/cc3CvITT.s:2594: Error: selected processor does not support `yield' in 
> ARM mode
> make[7]: *** [CMakeFiles/rocksdb.dir/build.make:3436: 
> CMakeFiles/rocksdb.dir/third-party/folly/folly/synchronization/DistributedMutex.cpp.o]
>  Error 1

so probably there is some inline assembly that assumes ARMv6 or later and
does not account for armel's ARMv5-based baseline.

On mipsel, it looks like 32 bits of address space might not be enough,
so you might need to try the same tricks that e.g. webkit2gtk uses to
save address space:

> virtual memory exhausted: Cannot allocate memory
> make[3]: *** [src/msg/CMakeFiles/common-msg-objs.dir/build.make:93: 
> src/msg/CMakeFiles/common-msg-objs.dir/Message.cc.o] Error 1

I hope this helps,
smcv

Re: Help porting Ceph 16.2.6 to mips6el, mipsel and armel

[Bastian Blank]

On Sat, Nov 20, 2021 at 11:31:04AM +0100, Thomas Goirand wrote:
> What worries me the most is mips6el, where the linker says "undefined
> reference to `__atomic_load_16'" (and more like this). I don't
> understand because there really is a -latomic parameter to GCC when
> linking, so it should be working.

Not all architectures support sub-wordsize atomic operations.  The only
way to fix that is to use a larger type (long usually).

Bastian

-- 
Extreme feminine beauty is always disturbing.
-- Spock, "The Cloud Minders", stardate 5818.4

Help porting Ceph 16.2.6 to mips6el, mipsel and armel

[Thomas Goirand]

Hi,

Latest Ceph doesn't build on 3 arch:

https://buildd.debian.org/status/package.php?p=ceph&suite=experimental

(plus the unofficial ports...)

What worries me the most is mips6el, where the linker says "undefined
reference to `__atomic_load_16'" (and more like this). I don't
understand because there really is a -latomic parameter to GCC when
linking, so it should be working.

I'm not sure what happens with the other arch, though it's important
that at least the libs can build (so that Qemu can be linked with librbd
support).

Help from anyone, prior to upload to Unstable to replace Ceph 14.2.21,
would be very much appreciated.

Cheers,

Thomas Goirand (zigo)

Bug#994856: ITP: harmonpy -- An algorithm to help integrate high-dimensional datasets

[Diane Trout]

Package: wnpp
Severity: wishlist
Owner: Diane Trout 
X-Debbugs-Cc: debian-devel@lists.debian.org

* Package name: harmonpy
  Version : 0.0.5
  Upstream Author : Kamil Slowikowski 
* URL : https://github.com/slowkow/harmonypy
* License : GPL-3+
  Programming Lang: Python
  Description : An algorithm to help integrate high-dimensional datasets

 Harmony is an algorithm for integrating multiple high-dimensional datasets.
 .
 harmonypy is a port of the harmony R package by Ilya Korsunsky.


Harmonypy is a package to help integrate high-dimentional datasets, such as
found while doing single-cell RNA-seq.

The popular bionformatics package Scanpy can use harmonypy for helping
integrate multiple datatypes, and some of the scanpy tests require harmonpy to
run.

As a bioinformatics package it pretty clearly belongs with the debian-med team.

Diane Trout

want help with autopkgtest for your package?

[Antonio Terceiro]

Hi,

The Debian CI team would like to encourage and help more maintainers to
add autopkgtest to their packages. To that effect, we now have a
repository called autopkgtest-help on salsa, where we will take help
requests from maintainers working on autopkgtest for their packages:

https://salsa.debian.org/ci-team/autopkgtest-help/

If you want help, please go ahead and create an issue in there.  To
quote the repository README:

  Valid requests:

  - _"I want to add autopkgtest to package X. X is a tool that [...]
and it works by [...]. How should I approach testing it?"_

It's OK if you have no idea where to start. But at least try to
describe your package, what it does and how it works so we can try
to help you.

  - _"I started writing autopkgtest for X, here is my current work in
progress [link]. But I encountered problem Y. How to I move
forward?"_

If you already have an autopkgtest but is having trouble making it
work as you think it should, you can also ask here.

  Invalid requests:

  - _"Please write autopkgtest for my package X for me"_.

As with anything else in free software, please show appreciation for
other people's time, and do your own research first. If you pose
your question with enough details (see above) and make it
interesting, it _may_ be that whoever answers will write at least a
basic structure for you, but as the maintainer you are still the
expert in the package and what tests are relevant.

If you ask your question soon, you might get your answer recorded in
video: we are going to have a DebConf21 talk next month, where we I and
Paul Gevers (elbrus) will answer a few autopkgtest questions in video
for posterity.

Now, if you have experience enabling autopkgtest for you own packages,
please consider watching that repository there to help us help our
fellow maintainers.


signature.asc
Description: PGP signature

Re: Need help with Multi-Arch in systemd

[Helmut Grohne]

On Thu, Jul 15, 2021 at 08:08:13AM +0200, Helmut Grohne wrote:
> I also proposed a solution here, which is avoiding
> SystemCallArchitectures=native. Initially, that sounds like a
> maintenance nightmare until you notice that it can be solved on a
> technical level. We already have dh_installsystemd. How bad would it be
> to have dh_installsystemd change .service files and automatically
> replace =native with a concrete architecture (in arch:any packages
> only)? systemd would no longer detect the architecture of services from
> its own one but rather use the one documented by the package. The mixing
> of architectures that our earlier solution broke would now partially
> work. We'd fix one package and binNMU the pile.

I've discussed this with #systemd on libera.chat. That resulted in a new
approach: Letting systemd detect the architecture of the binary being
executed. We've reported that as a feature request at
https://github.com/systemd/systemd/issues/20277. I therefore propose to
temporarily ignore the SystemCallArchitectures issue on the Debian side
and see what upstream thinks about it.

The libsystemd-shared issue can be tackled independently.

Helmut

Re: automatic NEW processing [was Re: Need help with Multi-Arch in systemd]

[Joerg Jaspert]

On 16200 March 1977, Michael Lustfield wrote:

I do recall that the FTP masters would've been generally open to have 
such an auto-approver (but maybe I'm wrong), but that no-one stepped 
up 
yet to code it up?

A few of us came up with some proof of concept designs/models, but we
ultimately dropped the effort when it became clear the team wasn't 
interested
in such tools. We considered continuing with a tool that would work 
for
individual users reviewing their own work, but there just wasn't 
enough

interest for that either.


I'd be happy to help resurrect (the personal-use version of) the 
project/effort

if there's any chance I won't be working on it by myself...


The place for that feature to end up in is inside dak, obviously.
That feature should read its config (is it enabled? for the current 
package? with which config for it?) from the database (projectb).
And then just hook itself into the part of the uploading that redirects 
packages to NEW.


And then one could look if it gets implemented a bit more generic and 
not NEW specific, but able to do something like this for any policy 
queue. (As in, backports NEW, p-u, whatever). So, configurable per 
queue. For the actual feature and the affected packages.


Any MR going in that direction will be great.

--
bye, Joerg

Re: automatic NEW processing [was Re: Need help with Multi-Arch in systemd]

[Michael Lustfield]

On Wed, 14 Jul 2021 18:48:24 +0200
Philipp Kern  wrote:

> On 14.07.21 13:47, Michael Biebl wrote:
> > Am 14.07.21 um 12:59 schrieb Simon McVittie:  
> I do recall that the FTP masters would've been generally open to have 
> such an auto-approver (but maybe I'm wrong), but that no-one stepped up 
> yet to code it up?

A few of us came up with some proof of concept designs/models, but we
ultimately dropped the effort when it became clear the team wasn't interested
in such tools. We considered continuing with a tool that would work for
individual users reviewing their own work, but there just wasn't enough
interest for that either.

I'd be happy to help resurrect (the personal-use version of) the project/effort
if there's any chance I won't be working on it by myself...

Re: Need help with Multi-Arch in systemd

[Joerg Jaspert]

On 16194 March 1977, Simon McVittie wrote:


Would it be feasible for dak to have a list of binary package name
regexes mapped to a source package and a section/priority, and 
auto-accept

packages from the given source package that match the regex, assigning
the given section/priority, without manual action? That would let the
ftp team pre-approve src:systemd to ship /^libsystemd-shared-[0-9]+$/
in libs/optional, for example.


At some point somewhere I think it was already said, but in general: We 
love MRs, we are at https://salsa.debian.org/ftp-team/dak/ and something 
doing kind of auto-NEW processing is NOT out of the question.


Exact details have to be hashed out, but this is the wrong thread for 
that. While sometimes NEW can be annoying, it regularly catches bugs 
even if "only a small change in packaging" happened, so it does have 
some reason why its there. But yes, there are some candidates (hello 
kernel) that can make use of something with less humans involved.


Your example above doesn't sound too bad as a starter, though haven't 
put any further thought into it yet.


--
bye, Joerg

Re: Need help with Multi-Arch in systemd

[Helmut Grohne]

Hi Michael,

On Thu, Jul 15, 2021 at 12:22:59AM +0200, Michael Biebl wrote:
> You are right. Thinking more about this, splitting out libsystemd-shared as
> a Multi-Arch: same library will not help with
> SystemCallArchitectures=native, which is used by the services in
> systemd-{container,journal-remote,...}.

That is correct, but it actually goes beyond systemd-* using systemd.
Any other service can face the very same problem.

> So splitting out libsystemd-shared, while in theory a nice solution, is not
> the right one in this case. We really need to ensure that systemd and
> systemd-* are of of the same architecture.

We have two related issues at hand. One is the libsystemd-shared
architecture matching and the other is the SystemCallArchitectures
matching. You appear to imply that both issues need to be addressed by
one common solution. Sure that would be nice, but is it required? Maybe
these issues are not that related after all.

Let us for a moment assume that we could magically make
SystemCallArchitectures work by locking users to the same architecture
as systemd. That would be bad in terms for mixed multiarch systems and
cross grading, because you essentially lock all daemons to the same
architecture as systemd. You fix the problem, by removing flexibility.

I also proposed a solution here, which is avoiding
SystemCallArchitectures=native. Initially, that sounds like a
maintenance nightmare until you notice that it can be solved on a
technical level. We already have dh_installsystemd. How bad would it be
to have dh_installsystemd change .service files and automatically
replace =native with a concrete architecture (in arch:any packages
only)? systemd would no longer detect the architecture of services from
its own one but rather use the one documented by the package. The mixing
of architectures that our earlier solution broke would now partially
work. We'd fix one package and binNMU the pile.

So while these problems are related, I think that forcing the
architectures to equal is a suboptimal solution for
SystemCallArchitectures.

> I still think that my idea of having a ":arch" annotation as counterpart to
> ":any" would be the most elegant way to achieve this. But since this is only
> an idea and not implemented, it's not something I can make use of. Do you
> think there is a chance we could convince dpkg and apt maintainers to add
> support for that?

If you replace :arch with :$DEB_HOST_ARCH, it already works today with
apt and dpkg, but not dose. The question is not whether we can implement
that (it already is), but whether we want to endorse these semantics or
not.

> Alternatively, your idea of systemd-core/systemd got me thinking.
> While I don't like the prospect of moving all (conf)files and state from one
> package to another, maybe we can turn this idea around.
> 
> We introduce an empty systemd-native package, which is
> Architecture:linux-any but *not* Multi-Arch: foreign/same/allowed.
> 
> systemd and all systemd-* packages would depend on systemd-native.
> This would link the architecture of systemd and systemd-* together and
> ensure they are the same.

I think this technically works. We also have prior art for this. blas
temporarily added such a package as a measure to fix #760936.

> Would this work for your cross-compile use-case?

I don't think this would negatively affect cross compiling. It could
affect people who try to run mixed-architecture systems though.

Given Simon's and Guillem's replies, I presently favour fixing the NEW
processing to package the library separately and fix
SystemCallArchitectures using dh_installsystemd, because it is
technically sound and does not negatively impact multiarch use cases.

Should I file a bug about SystemCallArchitectures such that we can track
it somewhere?

Helmut

Re: Need help with Multi-Arch in systemd

[Michael Biebl]

Am 09.07.21 um 14:24 schrieb Helmut Grohne:

Now let's do something stupid. Rename systemd to systemd-core (taking
all files with it, please refrain from discussing the name unless you
seriously consider doing this). Mark it Multi-Arch: allowed. Add a new,
empty binary package systemd. It is Multi-Arch: foreign and depends on
systemd-core:any. This approach would technically satisfy all three
requirements, but it feels a little crazy to me.


[..]


And I fear we have another related issue that we swept under the carpet
thus far. man systemd.exec explains SystemCallArchitectures=native.



You are right. Thinking more about this, splitting out libsystemd-shared 
as a Multi-Arch: same library will not help with 
SystemCallArchitectures=native, which is used by the services in 
systemd-{container,journal-remote,...}.
So splitting out libsystemd-shared, while in theory a nice solution, is 
not the right one in this case. We really need to ensure that systemd 
and systemd-* are of of the same architecture.


I still think that my idea of having a ":arch" annotation as counterpart 
to ":any" would be the most elegant way to achieve this. But since this 
is only an idea and not implemented, it's not something I can make use 
of. Do you think there is a chance we could convince dpkg and apt 
maintainers to add support for that?



Alternatively, your idea of systemd-core/systemd got me thinking.
While I don't like the prospect of moving all (conf)files and state from 
one package to another, maybe we can turn this idea around.


We introduce an empty systemd-native package, which is 
Architecture:linux-any but *not* Multi-Arch: foreign/same/allowed.


systemd and all systemd-* packages would depend on systemd-native.
This would link the architecture of systemd and systemd-* together and 
ensure they are the same.

Would this work for your cross-compile use-case?

Michael



OpenPGP_signature
Description: OpenPGP digital signature

Re: Need help with Multi-Arch in systemd

[Guillem Jover]

On Wed, 2021-07-14 at 11:59:11 +0100, Simon McVittie wrote:
> On Thu, 08 Jul 2021 at 23:03:48 +0200, Michael Biebl wrote:
> > [a separate libsystemd-shared-249 .deb] would also mean, that on every
> > new upstream release, systemd would have to go through NEW
> 
> It seems like we're rejecting a good technical solution because
> social/organisational factors block it (namely, new binary packages
> triggering manual review from the ftp team even if there has not been
> any significant change in how the package is organised, resulting in
> manual review being artificially frequent for libraries that happen to
> break ABI a lot, but infrequent for packages that aren't libraries or
> don't break ABI).

Yes. I was mentioning exactly this the other day on the
#debian-bootstrap IRC channel.

In addition having this automatic support could make life easier for
many other potential packages.

> This seems like a side-effect of the ftp team's two gatekeeping roles
> (legal review and managing the namespaces of source and binary package
> names) both having the dak override file as their implementation, rather
> than necessarily anything that was designed or intended.

Yes, plus section and priority-spaces. But then, I don't see why a binary
package rename should trigger a new legal audit.

> Would it be feasible for dak to have a list of binary package name
> regexes mapped to a source package and a section/priority, and auto-accept
> packages from the given source package that match the regex, assigning
> the given section/priority, without manual action? That would let the
> ftp team pre-approve src:systemd to ship /^libsystemd-shared-[0-9]+$/
> in libs/optional, for example.

What I had in mind was that DAK would gain support for automatic
ACCEPT of binary package renames due to SONAME version bumps,
something like this:

  * If the new bin:lib:
- replaces an existing bin:lib from the same
  source, where  is lower than .
- contains a shared library mapping to that package name.
- is in section */libs or */oldlibs.
  * Then → auto-ACCEPT, pre-filling the new section/priority from the
old binary package.
  * Otherwise → NEW.

I guess it could potentially be further extended later on to cover
other safe non shared library cases.

But if that's too much to ask, either due to implementation or policy
concerns, I'd take an explicit allowlist letting specific cases
through, such as the systemd one, instead of having to settle for
either suboptimal or wrong solutions for the problem at hand, due
to the currently required workflow being too cumbersome.

Thanks,
Guillem

Re: automatic NEW processing [was Re: Need help with Multi-Arch in systemd]

[Philipp Kern]

On 14.07.21 13:47, Michael Biebl wrote:

Am 14.07.21 um 12:59 schrieb Simon McVittie:

Would it be feasible for dak to have a list of binary package name
regexes mapped to a source package and a section/priority, and 
auto-accept

packages from the given source package that match the regex, assigning
the given section/priority, without manual action? That would let the
ftp team pre-approve src:systemd to ship /^libsystemd-shared-[0-9]+$/
in libs/optional, for example.

It seems like this would also be good for src:linux, where ABI breaks
are often tied to security fixes that should enter the archive ASAP.


If something fully automated like this would be implemented, I would 
have much less concerns with this option.


As it stands today, NEW processing is simply to unpredictable. It can 
range from taking a a few hours/days to several months.


And yet it should not dictate technical solutions. We basically see the 
same thing with nvidia-graphics-drivers that break your running 
applications when the libraries are upgraded and you don't reboot. 
Arguably the proper solution is to version them with the full 
major/minor version. But I can see how that's a total hassle with NEW 
processing for both for the maintainer and to the FTP team.


I do recall that the FTP masters would've been generally open to have 
such an auto-approver (but maybe I'm wrong), but that no-one stepped up 
yet to code it up?


Kind regards
Philipp Kern

Re: Need help with Multi-Arch in systemd

[Bastian Blank]

On Wed, Jul 14, 2021 at 11:59:11AM +0100, Simon McVittie wrote:
> It seems like this would also be good for src:linux, where ABI breaks
> are often tied to security fixes that should enter the archive ASAP.

As security updates are hand approved, accepting by NEW does not help
that much.

Bastian

-- 
Peace was the way.
-- Kirk, "The City on the Edge of Forever", stardate unknown

Re: Need help with Multi-Arch in systemd

[Bastian Blank]

On Thu, Jul 08, 2021 at 11:03:48PM +0200, Michael Biebl wrote:
> Asking on #debian-systemd, Marco d'Itri suggested, that we move
> libsystemd-shared into a separate binary package. This would only help, if
> we moved libsystemd-shared into a Multi-Arch location (which means, we'd
> have to carry a patch against upstream). It would also mean, that on every
> new upstream release, systemd would have to go through NEW.
> So I'm not very keen on doing that.

Why do you think?  libsystemd-shared is a perfectly valid package name
and it does not change.

However, if you call it libsystemd-shared-249, it is supposed to be
stable.

Bastian

-- 
Is truth not truth for all?
-- Natira, "For the World is Hollow and I have Touched
   the Sky", stardate 5476.4.

Re: Need help with Multi-Arch in systemd

[Timo Röhling]

* Simon McVittie  [2021-07-14 11:59]:

Would it be feasible for dak to have a list of binary package name
regexes mapped to a source package and a section/priority, and auto-accept
packages from the given source package that match the regex, assigning
the given section/priority, without manual action? That would let the
ftp team pre-approve src:systemd to ship /^libsystemd-shared-[0-9]+$/
in libs/optional, for example.

+1

Especially considering that library packages are subject to transitions
anyway, so they already receive much more scrutiny than, say, an
updated Python module.

Cheers
Timo

--
⢀⣴⠾⠻⢶⣦⠀   ╭╮
⣾⠁⢠⠒⠀⣿⡁   │ Timo Röhling   │
⢿⡄⠘⠷⠚⠋⠀   │ 9B03 EBB9 8300 DF97 C2B1  23BF CC8C 6BDD 1403 F4CA │
⠈⠳⣄   ╰╯


signature.asc
Description: PGP signature

Re: automatic NEW processing [was Re: Need help with Multi-Arch in systemd]

[Michael Biebl]

Am 14.07.21 um 13:47 schrieb Michael Biebl:

Am 14.07.21 um 12:59 schrieb Simon McVittie:

Would it be feasible for dak to have a list of binary package name
regexes mapped to a source package and a section/priority, and 
auto-accept

packages from the given source package that match the regex, assigning
the given section/priority, without manual action? That would let the
ftp team pre-approve src:systemd to ship /^libsystemd-shared-[0-9]+$/
in libs/optional, for example.

It seems like this would also be good for src:linux, where ABI breaks
are often tied to security fixes that should enter the archive ASAP.


If something fully automated like this would be implemented, I would 
have much less concerns with this option.



Fwiw, I like your proposal and would like to see it implemented 
regardless of the issue at hand as I think it could be generally useful.


Thanks, Simon!




OpenPGP_signature
Description: OpenPGP digital signature

automatic NEW processing [was Re: Need help with Multi-Arch in systemd]

[Michael Biebl]

Am 14.07.21 um 12:59 schrieb Simon McVittie:

Would it be feasible for dak to have a list of binary package name
regexes mapped to a source package and a section/priority, and auto-accept
packages from the given source package that match the regex, assigning
the given section/priority, without manual action? That would let the
ftp team pre-approve src:systemd to ship /^libsystemd-shared-[0-9]+$/
in libs/optional, for example.

It seems like this would also be good for src:linux, where ABI breaks
are often tied to security fixes that should enter the archive ASAP.


If something fully automated like this would be implemented, I would 
have much less concerns with this option.


As it stands today, NEW processing is simply to unpredictable. It can 
range from taking a a few hours/days to several months.


Regards,
Michael









OpenPGP_signature
Description: OpenPGP digital signature

Re: Need help with Multi-Arch in systemd

[Simon McVittie]

On Thu, 08 Jul 2021 at 23:03:48 +0200, Michael Biebl wrote:
> [a separate libsystemd-shared-249 .deb] would also mean, that on every
> new upstream release, systemd would have to go through NEW

It seems like we're rejecting a good technical solution because
social/organisational factors block it (namely, new binary packages
triggering manual review from the ftp team even if there has not been
any significant change in how the package is organised, resulting in
manual review being artificially frequent for libraries that happen to
break ABI a lot, but infrequent for packages that aren't libraries or
don't break ABI).

This seems like a side-effect of the ftp team's two gatekeeping roles
(legal review and managing the namespaces of source and binary package
names) both having the dak override file as their implementation, rather
than necessarily anything that was designed or intended.

Would it be feasible for dak to have a list of binary package name
regexes mapped to a source package and a section/priority, and auto-accept
packages from the given source package that match the regex, assigning
the given section/priority, without manual action? That would let the
ftp team pre-approve src:systemd to ship /^libsystemd-shared-[0-9]+$/
in libs/optional, for example.

It seems like this would also be good for src:linux, where ABI breaks
are often tied to security fixes that should enter the archive ASAP.

smcv

Re: Need help with Multi-Arch in systemd

[Josh Triplett]

Helmut Grohne wrote:
> So you made me thinking, can we somehow implement this with our
> current spec? The most important requirements seem to be:
>
>  * libsystemd-shared.so and /sbin/systemd need to reside in the same
>binary package.
>  * It shall be possible to depend on libsystemd-shared.so for a
>particular architecture.
>  * A dependency on "systemd" should request a native systemd.
>
> Now let's do something stupid. Rename systemd to systemd-core (taking
> all files with it, please refrain from discussing the name unless you
> seriously consider doing this). Mark it Multi-Arch: allowed. Add a new,
> empty binary package systemd. It is Multi-Arch: foreign and depends on
> systemd-core:any. This approach would technically satisfy all three
> requirements, but it feels a little crazy to me.

This seems like a rather reasonable approach, actually. It's a little
unusual, but it has all the advantages of making systemd multi-arch
aware while not creating the trap of making a `systemd` dependency do
the wrong thing, because `systemd` has the right multi-arch dependency
on `systemd-core:any`.

The handful of packages that really do need a same-arch dependency on
systemd (those that are part of systemd itself and need
libsystemd-shared) could depend on `systemd-core` directly, and
everything else can continue depending on systemd with no transition
required. And there's only one trip through NEW, once.

Re: Need help with Multi-Arch in systemd

[Bálint Réczey]

Hi Michael,

Michael Biebl  ezt írta (időpont: 2021. júl. 9., P, 22:42):
>
> Hi Guillem,
>
> thanks for your feedback
>
> Am 09.07.21 um 13:46 schrieb Guillem Jover:
> > If the private library has no backwards or forward compatibility (due
> > to the SONAME used) the time window where the library does not match
> > the expectations of the stuff linked to it might indeed be too big.
>
> The libsystemd-shared library is considered an implementation detail and
> indeed has no guaranteed backwards or forward compatibility.
> The soname is bumped when the first rc1 release is made (e.g v249-rc1 →
> libsystemd-shared-249.so) and there might even be incompatible changes
> between the rc1 and the final release.
>
> > But the reported bug is just a symptom of a bigger issue. This problem
> > already exists for any of the other binary packages built against this
> > private shared library, there's a time-window where they will not work
> > if the installation gets interrupted or fails, compared with public
> > shared libraries.
>
> This observation is correct I'd say. Currently we have the following
> split-off binary packages which ship binaries that link against
> libsystemd-shared:
>
> systemd-container (4 binaries in $PATH, 7 services, 11 total)
> systemd-coredump (1 binary in $PATH, 1 service, 2 total)
> systemd-journal-remote (3 services, 3 total)
> systemd-timesyncd (1 service, 1 total)
>
> (I'll exclude systemd-tests, as this is a special case)
>
> The main difference here, is that none of those binaries is really
> critical for the runtime of the system.
> An unbootable system though is one of the worst things that can happen.
> Which is why I'm really reluctant to split off libsystemd-shared from
> systemd and hopefully also explains why in general I'm not super keen on
> chopping up src:systemd unnecessarily.
>
> > This implies to me the correct solution is a name-versioned package,
> > even though that seems cumbersome given our current archive handling
> > practices this is IMO the only correct solution.
>
> A name-versioned package would certainly be better then an unversioned
> libsystemd-shared package. It still would make PID 1 a bit more brittle
> though (see e.g. my comment regard rc releases).
> It would also need patching of the build system, to override the rpath,
> which would have to be multi-arch aware.
>
> $ find . -name meson.build | xargs grep rpath | wc -l
> 123
>
> This would be a significant patch with a lot of ongoing churn and
> maintenance effort. I'm not sure if I'm willing or even able to do that.

IMO what Guillem recommends, i.e. the name-versioned libsystemd
package with versioned shared library name is still the cleanest and
so far the only reasonably reliable solution. IMO going through NEW in
every few months could be an acceptable cost especially since I was
quite happy with the pace at which NEW is processed recently. I don't
maintain systemd in Ubuntu anymore, but if I were, I'd be happy to
accept this increased maintenance burden for the sake of having a
clean solution for the Multi-Arch problem. If release candidates will
be uploaded only to experimental then the incompatible changes won't
cause problems. Even if RC-s are uploaded to unstable and a rarely
occurring incompatible change takes place then the library package
name can be bumped again.
After we discussed this topic on #debian devel on 2020-05-05 I even
started implementing the solution but did not finish it because it was
not an important problem from Ubuntu's POV.

I think the rpath usage search counted a lot of hits which don't have
to be changed and maybe upstream would be willing to accept the patch.

Cheers,
Balint

Re: Need help with Multi-Arch in systemd

[Helmut Grohne]

Hi Michael,

I'm not yet fully convinced that we're out of options, but let's for a
moment assume we were.

On Fri, Jul 09, 2021 at 10:26:43PM +0200, Michael Biebl wrote:
> So, unless we get a :arch annotation that can be used for M-A:foreign
> packages, maybe the best option is

Given Johannes' reply, I think that :arch annotations can be a strict
improvement to the status quo. dose will ignore them, so you aren't
worse off with it. apt and dpkg agree to honour them, so they'd
practically solve the symptoms (despite leading dose into non-solutions
to certain satisfiability issues that are mostly relevant to cross
building only).

The one question here is whether those are the semantics that we want
for this corner case of multiarch. A while ago, David changed apt to
make it stop behaving like dose and instead make it behave like dpkg.
That looks like we had consensus back then. Did we document it anywhere?

> option 5)
> do nothing?
> 
> I mean, we shipped the M-A: foreign notation for systemd since 2014.
> If there is one bug report regarding such an architecture mismatch in 7
> years, maybe the most pragmatic approach is to simply ignore it, given the
> downsides of the alternatives? I'm not sure what the best course of action
> is here.

If :arch qualifiers are an option, I think they are a strict improvement
over doing nothing. That's something I couldn't convincingly say about
any other option. In any case, I don't think fixing this in time for
bullseye is reasonable, so we do have some time figuring out whether
:arch is a solution.

Helmut

Re: Need help with Multi-Arch in systemd

[Adrian Bunk]

On Fri, Jul 09, 2021 at 10:28:57AM +0200, Helmut Grohne wrote:
>...
> I also disagree with the need to go through NEW more than once. The new
> package could quite simply be named libsystemd-private and lack a
> .symbols and .shlibs file. Internal users would always use (=
> ${binary:Version}) anyway. The package description would declare that
> any external dependency on libsystemd-private is a bug. Prior art:
> libbinutils/binutils-dev.
>...

The prior art of libbinutils/binutils-dev is that instead of shared 
linking, users are linking to the static version to the private library
(see e.g. #964536).

I don't understand what benefits this switch to static linking gives,
except for making security support harder.

> Helmut

cu
Adrian

Re: Need help with Multi-Arch in systemd

[Sam Hartman]

> "Helmut" == Helmut Grohne  writes:

Helmut> Looks like this leaves us between a rock and a hard
Helmut> place. None of the options seems particularly attractive to
Helmut> me at this point.

We seem to be in a brainstorming space here, throwing out half baked
ideas because none of us has a great answer.  So, I'm going to join the
party.  Are there games we could play with prerm scripts in a
systemd-private package to make sure that the old systemd private
library sticks around until the upgrade finishes when upgrading
systemd-private?

--Sam

Re: Need help with Multi-Arch in systemd

[Johannes Schauer Marin Rodrigues]

Quoting Helmut Grohne (2021-07-09 14:24:01)
> Another possibility (kudos to David Kalnischkies) would be exploiting
> something I might call a loophole in the Multi-Arch spec. While we don't
> currently use arch-qualified dependencies in the archive, the spec considers
> them and dpkg and apt support them (somewhat). So in theory, we could say
> that systemd-container Depends: systemd:${DEB_HOST_ARCH}.  I'm not sure
> whether all resolvers agree on this, but I'd expect this dependency to pull
> the right instance (despite being marked Multi-Arch: foreign). Before jumping
> onto this, please consider David's warning: here be dragons.

No, not all solvers agree on this. Imagine this situation:

Package: pkga
Architecture: amd64
Depends: pkgb:i386
Multi-Arch: no

Package: pkgb
Architecture: i386
Multi-Arch: foreign

dose3, apt and dpkg agree that pkga can be installed because its dependency is
satisfied by pkgb. But if we change pkgb:i386 to pkgb:amd64 to end up with
this:

Package: pkga
Architecture: amd64
Depends: pkgb:amd64
Multi-Arch: no

Package: pkgb
Architecture: i386
Multi-Arch: foreign

Then dose3 can satisfy the dependency of pkg while apt and dpkg cannot. This is
because when building the cudf representation of above two packages, dose3
considers pkg to provide pkgb for all architectures, including amd64, because
it is marked as m-a:foreign.

Thanks!

cheers, josch

signature.asc
Description: signature

Re: Need help with Multi-Arch in systemd

[Michael Biebl]

Hi Guillem,

thanks for your feedback

Am 09.07.21 um 13:46 schrieb Guillem Jover:

If the private library has no backwards or forward compatibility (due
to the SONAME used) the time window where the library does not match
the expectations of the stuff linked to it might indeed be too big.


The libsystemd-shared library is considered an implementation detail and 
indeed has no guaranteed backwards or forward compatibility.
The soname is bumped when the first rc1 release is made (e.g v249-rc1 → 
libsystemd-shared-249.so) and there might even be incompatible changes 
between the rc1 and the final release.



But the reported bug is just a symptom of a bigger issue. This problem
already exists for any of the other binary packages built against this
private shared library, there's a time-window where they will not work
if the installation gets interrupted or fails, compared with public
shared libraries.


This observation is correct I'd say. Currently we have the following 
split-off binary packages which ship binaries that link against 
libsystemd-shared:


systemd-container (4 binaries in $PATH, 7 services, 11 total)
systemd-coredump (1 binary in $PATH, 1 service, 2 total)
systemd-journal-remote (3 services, 3 total)
systemd-timesyncd (1 service, 1 total)

(I'll exclude systemd-tests, as this is a special case)

The main difference here, is that none of those binaries is really 
critical for the runtime of the system.
An unbootable system though is one of the worst things that can happen. 
Which is why I'm really reluctant to split off libsystemd-shared from 
systemd and hopefully also explains why in general I'm not super keen on 
chopping up src:systemd unnecessarily.



This implies to me the correct solution is a name-versioned package,
even though that seems cumbersome given our current archive handling
practices this is IMO the only correct solution.


A name-versioned package would certainly be better then an unversioned 
libsystemd-shared package. It still would make PID 1 a bit more brittle 
though (see e.g. my comment regard rc releases).
It would also need patching of the build system, to override the rpath, 
which would have to be multi-arch aware.


$ find . -name meson.build | xargs grep rpath | wc -l
123

This would be a significant patch with a lot of ongoing churn and 
maintenance effort. I'm not sure if I'm willing or even able to do that.



Another solution might be to request upstream to stabilize this library?


The point of libsystemd-shared is to be an internal implementation 
detail where ABI can be broken. There are certainly no plans to change 
that upstream and I don't think I can convince upstream otherwise on 
this matter.



Yet another solution, which seems suboptimal, might be to statically
link it, but the shared library seems rather big, but perhaps if the
programs linked only use parts of it, that might not be too bad?


So, option 4).
I tried that for v249. It's doable, but as this there no upstream 
support for that, it requires a patch which is significant and would 
require ongoing maintenance effort as well.


Regarding the numbers:

systemd-container (11 binaries)
Installed-Size: [-1278-] {+5644+}

systemd-coredump (2 binaries)
Installed-Size: [-276-] {+1106+}

systemd-journal-remote (3 binaries)
Installed-Size: [-336-] {+1218+}

systemd-timesyncd (1 binary)
Installed-Size: [-209-] {+595+}

I think the increase in size is significant.
More importantly, I'd need help maintaining this patch going forward


I acknowledge, that option 2 and 3 makes Helmut's work on cross-building 
harder, although I think that option 3 would be preferrable to 2, as it 
as least leaves the door open for making cross-building possible.



So, unless we get a :arch annotation that can be used for M-A:foreign 
packages, maybe the best option is


option 5)
do nothing?

I mean, we shipped the M-A: foreign notation for systemd since 2014.
If there is one bug report regarding such an architecture mismatch in 7 
years, maybe the most pragmatic approach is to simply ignore it, given 
the downsides of the alternatives? I'm not sure what the best course of 
action is here.


Regards,
Michael



OpenPGP_signature
Description: OpenPGP digital signature

Re: Need help with Multi-Arch in systemd

[Timo Röhling]

* Michael Biebl  [2021-07-09 13:24]:

Let me be blunt here: I'm not willing to compromise on robustness here.

Well, I have had my say and ultimately, it's your package and
your decision, of course.

Cheers
Timo

--
⢀⣴⠾⠻⢶⣦⠀   ╭╮
⣾⠁⢠⠒⠀⣿⡁   │ Timo Röhling   │
⢿⡄⠘⠷⠚⠋⠀   │ 9B03 EBB9 8300 DF97 C2B1  23BF CC8C 6BDD 1403 F4CA │
⠈⠳⣄   ╰╯


signature.asc
Description: PGP signature

Re: Need help with Multi-Arch in systemd

[Helmut Grohne]

Hi Michael,

in your other mail, you gave a very good reason for keeping the systemd
binary and libsystemd-shared.so in the same binary package. That
improves my understanding of why you favour option 3.

On Fri, Jul 09, 2021 at 11:31:15AM +0200, Michael Biebl wrote:
> Am 09.07.21 um 10:28 schrieb Helmut Grohne:
> > 
> > On Thu, Jul 08, 2021 at 11:03:48PM +0200, Michael Biebl wrote:
> 
> > > Option 2 would be to drop Multi-Arch: foreign from systemd. This would 
> > > mean,
> > > that packages like libpam-systemd/libnss-systemd can no longer be 
> > > installed
> > > for a foreign architecture (even though those modules only use 
> > > architecture
> > > independent interfaces of systemd).
> > 
> > That would break a pile of use cases and cause a lot of pain downstream.
> > Please don't.
> 
> Can you iterate on that a bit? I was looking for use cases, but couldn't
> come up with them.

I guess you already know what comes next: cross building. 4700 packages
include systemd in their Build-Depends. Many of them would immediately
become cross bd-uninstallable. For others, the installation set would be
satisfied with a foreign systemd, which would quite simply fail to work.

Any kind of mixed-arch system where you want to install a daemon that
depends on systemd for a foreign architecture would also be affected.
Someone attempting to do so might accidentally cross grade systemd in
doing so.

> As said, option 1 is not something I really consider a viable option for the
> aforementioned reasons.

>From my pov, the only reason left against 1 is partial upgrades breaking
the installation. It's a strong one anyhow.

> So, packages need to decide whether they use architecture independent
> interfaces of systemd or not and annotate it accordingly. But they only need
> to do that once. Isn't this the same as with any python, ruby or perl
> dependency?

Yes. For perl and python, we haven't even annotated half of the relevant
Depends with :any yet. Experience tells, the annotation is not gonna
happen.

> Maybe having something like the inverse of M-A: allowed would be useful?
> Instead of having packages explicitly opt-in via a :any annotation, they can
> opt out and request the same architecture even if a package is marked as
> M-A: foreign?
> So the select few cases which use an architecture dependent interface of
> M-A: foreign package would use something like pkg:arch ?

If I were to design Multi-Arch today, I would likely agree. The
transition cost to get there seems prohibitively high though. I've tried
hard to change the spec and I did not succeed (on the Multi-Arch
interpreter problem). Yet, we devised a workaround for that issue. So
you made me thinking, can we somehow implement this with our current
spec? The most important requirements seem to be:

 * libsystemd-shared.so and /sbin/systemd need to reside in the same
   binary package.
 * It shall be possible to depend on libsystemd-shared.so for a
   particular architecture.
 * A dependency on "systemd" should request a native systemd.

Now let's do something stupid. Rename systemd to systemd-core (taking
all files with it, please refrain from discussing the name unless you
seriously consider doing this). Mark it Multi-Arch: allowed. Add a new,
empty binary package systemd. It is Multi-Arch: foreign and depends on
systemd-core:any. This approach would technically satisfy all three
requirements, but it feels a little crazy to me.

Another possibility (kudos to David Kalnischkies) would be exploiting
something I might call a loophole in the Multi-Arch spec. While we don't
currently use arch-qualified dependencies in the archive, the spec
considers them and dpkg and apt support them (somewhat). So in theory,
we could say that systemd-container Depends: systemd:${DEB_HOST_ARCH}.
I'm not sure whether all resolvers agree on this, but I'd expect this
dependency to pull the right instance (despite being marked Multi-Arch:
foreign). Before jumping onto this, please consider David's warning:
here be dragons.

I'm relaying another question from David: Assuming we'd solve the
libsystemd-shared.so issue, would systemd-container:arch1 actually work
with systemd:arch2 or are there more aspects where the same architecture
is assumed?

As we already are going crazy in the search for solutions to this, let
me propose a scary one. I have no idea whether this works in practice.
Split out libsystemd-shared.so into systemd-private like before, but at
the same time ship it in systemd. That causes a conflict of course, so
systemd-private Conflicts: systemd:${DEB_HOST_ARCH} and systemd
Conflicts: systemd-private:${DEB_HOST_ARCH} as well as Provides:
systemd-private:${DEB_HOST_ARCH}. These Conflicts must be arch-qualified
as otherwise they're interpreted as any instance. We already have
arch-qualified Conflicts in the archive, so this aspect likely works.
I'm less sure about arch-qualified Provides though. In any case, the
(common) native case would be systemd without systemd-private 

Re: Need help with Multi-Arch in systemd

[Guillem Jover]

On Fri, 2021-07-09 at 12:29:19 +0200, Michael Biebl wrote:
> Am 09.07.2021 um 11:37 schrieb Timo Röhling:
> > * Michael Biebl  [2021-07-09 11:01]:
> > > Splitting out libsystemd-shared (once) will make PID1 very brittle.
> > > It can lead to situations where old systemd + new libsystemd-private
> > > is installed. If the installation is aborted at this point, you have
> > > an unbootable system.

> > Helmut suggested a tightly versioned dependency, so such a state
> > would never be consistent.

> That tightly versioned dependency doesn't help unfortunately. There is still
> a time window between the new libsystemd-shared and the new systemd being
> unpacked.

If the private library has no backwards or forward compatibility (due
to the SONAME used) the time window where the library does not match
the expectations of the stuff linked to it might indeed be too big.
But the reported bug is just a symptom of a bigger issue. This problem
already exists for any of the other binary packages built against this
private shared library, there's a time-window where they will not work
if the installation gets interrupted or fails, compared with public
shared libraries.

This implies to me the correct solution is a name-versioned package,
even though that seems cumbersome given our current archive handling
practices this is IMO the only correct solution. The alternative of
using Multi-Arch:allowed is a workaround for this packaging problem.

Another solution might be to request upstream to stabilize this library?
Yet another solution, which seems suboptimal, might be to statically
link it, but the shared library seems rather big, but perhaps if the
programs linked only use parts of it, that might not be too bad?

Thanks,
Guillem

Re: Need help with Multi-Arch in systemd

[Michael Biebl]

Am 09.07.2021 um 13:01 schrieb Timo Röhling:

* Michael Biebl  [2021-07-09 12:29]:
That tightly versioned dependency doesn't help unfortunately. There is 
still a time window between the new libsystemd-shared and the new 
systemd being unpacked.

There's also a time window between files in /usr/bin and files in
/usr/lib being replaced.

If you reboot your system with an unfinished/interrupted upgrade,
you cannot assume a consistent or bootable state. Packaging all
stuff in the same package *may* reduce the above time window
somewhat, but it will not eliminate the underlying problem, so it is
a non-solution.

There may be other reasons to object Helmut's proposal, but this is
not it.


I don't agree.
The time window between dpkg unpacking a single package and ending up in 
a inconsistent state is neglibible to apt updating multiple packages 
during a dist-upgrade.


Let me be blunt here: I'm not willing to compromise on robustness here.

Re: Need help with Multi-Arch in systemd

[Timo Röhling]

* Michael Biebl  [2021-07-09 12:29]:
That tightly versioned dependency doesn't help unfortunately. There is 
still a time window between the new libsystemd-shared and the new 
systemd being unpacked.

There's also a time window between files in /usr/bin and files in
/usr/lib being replaced.

If you reboot your system with an unfinished/interrupted upgrade,
you cannot assume a consistent or bootable state. Packaging all
stuff in the same package *may* reduce the above time window
somewhat, but it will not eliminate the underlying problem, so it is
a non-solution.

There may be other reasons to object Helmut's proposal, but this is
not it.

Cheers
Timo

--
⢀⣴⠾⠻⢶⣦⠀   ╭╮
⣾⠁⢠⠒⠀⣿⡁   │ Timo Röhling   │
⢿⡄⠘⠷⠚⠋⠀   │ 9B03 EBB9 8300 DF97 C2B1  23BF CC8C 6BDD 1403 F4CA │
⠈⠳⣄   ╰╯


signature.asc
Description: PGP signature

Re: Need help with Multi-Arch in systemd

[Michael Biebl]

Am 09.07.2021 um 11:37 schrieb Timo Röhling:

* Michael Biebl  [2021-07-09 11:01]:
Splitting out libsystemd-shared (once) will make PID1 very brittle. It 
can lead to situations where old systemd + new libsystemd-private is 
installed. If the installation is aborted at this point, you have an 
unbootable system.

Helmut suggested a tightly versioned dependency, so such a state
would never be consistent.


That tightly versioned dependency doesn't help unfortunately. There is 
still a time window between the new libsystemd-shared and the new 
systemd being unpacked.


Michael

Re: Need help with Multi-Arch in systemd

[Timo Röhling]

* Michael Biebl  [2021-07-09 11:01]:
Splitting out libsystemd-shared (once) will make PID1 very brittle. It 
can lead to situations where old systemd + new libsystemd-private is 
installed. If the installation is aborted at this point, you have an 
unbootable system.

Helmut suggested a tightly versioned dependency, so such a state
would never be consistent. And given that system updates are not
atomic transactions, I'm pretty confident you can interrupt *any*
update of a boot-critical package at a point that leaves the system
unbootable.

Cheers
Timo

--
⢀⣴⠾⠻⢶⣦⠀   ╭╮
⣾⠁⢠⠒⠀⣿⡁   │ Timo Röhling   │
⢿⡄⠘⠷⠚⠋⠀   │ 9B03 EBB9 8300 DF97 C2B1  23BF CC8C 6BDD 1403 F4CA │
⠈⠳⣄   ╰╯


signature.asc
Description: PGP signature

Re: Need help with Multi-Arch in systemd

[Michael Biebl]

Am 09.07.21 um 10:28 schrieb Helmut Grohne:


On Thu, Jul 08, 2021 at 11:03:48PM +0200, Michael Biebl wrote:



Option 2 would be to drop Multi-Arch: foreign from systemd. This would mean,
that packages like libpam-systemd/libnss-systemd can no longer be installed
for a foreign architecture (even though those modules only use architecture
independent interfaces of systemd).


That would break a pile of use cases and cause a lot of pain downstream.
Please don't.


Can you iterate on that a bit? I was looking for use cases, but couldn't 
come up with them.



Option 3 is something I came up with after reading the Multi-Arch related
wiki pages:
https://salsa.debian.org/systemd-team/systemd/-/commit/c379461a14dcd13e0b625bd8faabbcf7fb3d19d6
It would mean marking systemd as Multi-Arch: allowed. And packages that only
use architecture interfaces of systemd would have to use the :any
annotation.


Yeah, this technically works, but it causes a lot of maintenance effort,
because you get to hunt down every single systemd dependency in the
archive for years. I think our time is worth more than the cost of
introducing a new binary package into the archive.


As said, option 1 is not something I really consider a viable option for 
the aforementioned reasons.

I would appreciate feedback, if option 3 is proper solution or not, or if
there are other, better alternatives. If we go with option 3, should I
inform all rdeps of systemd to update their dependency accordingly, i.e. do
a MBF?


The problem I see here is less with annotating all deps once. It is more
the constant effort it incurs. It is not obvious that you should
annotate your systemd dependency :any. People will forget that when
adding systemd dependencies. What I take issue with is the permanent
cost that we keep in the long run.


So, packages need to decide whether they use architecture independent 
interfaces of systemd or not and annotate it accordingly. But they only 
need to do that once. Isn't this the same as with any python, ruby or 
perl dependency?


Maybe having something like the inverse of M-A: allowed would be useful?
Instead of having packages explicitly opt-in via a :any annotation, they 
can opt out and request the same architecture even if a package is 
marked as M-A: foreign?
So the select few cases which use an architecture dependent interface of 
M-A: foreign package would use something like pkg:arch ?













OpenPGP_signature
Description: OpenPGP digital signature

Re: Need help with Multi-Arch in systemd

[Michael Biebl]

Am 09.07.21 um 10:28 schrieb Helmut Grohne:

I concur with Marco here and I argue that splitting the library is my
preferred solution. I confirm that you'd need to move the library for
this to work. For the other points I do not follow. I think it would be
ok to move the library using code inside debian/. Is there any reason
why you ruled out that option?

I also disagree with the need to go through NEW more than once. The new
package could quite simply be named libsystemd-private and lack a
.symbols and .shlibs file.


Splitting out libsystemd-shared (once) will make PID1 very brittle. It 
can lead to situations where old systemd + new libsystemd-private is 
installed. If the installation is aborted at this point, you have an 
unbootable system.

This is why I think it's not a viable approach.




OpenPGP_signature
Description: OpenPGP digital signature

Re: Need help with Multi-Arch in systemd

[Helmut Grohne]

Hi Michael,

thanks for reaching out!

On Thu, Jul 08, 2021 at 11:03:48PM +0200, Michael Biebl wrote:
> a couple of days ago, the following bug report was filed against systemd
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=990547

Imo, this is bug should be serious. In essence, it is a missing
dependency and we track missing dependencies as serious.

At the same time, I find it so unlikely to happen in practice combined
with the difficulty of finding a good solution that I'd propose
bullseye-ignore.

> Asking on #debian-systemd, Marco d'Itri suggested, that we move
> libsystemd-shared into a separate binary package. This would only help, if
> we moved libsystemd-shared into a Multi-Arch location (which means, we'd
> have to carry a patch against upstream). It would also mean, that on every
> new upstream release, systemd would have to go through NEW.
> So I'm not very keen on doing that.

I concur with Marco here and I argue that splitting the library is my
preferred solution. I confirm that you'd need to move the library for
this to work. For the other points I do not follow. I think it would be
ok to move the library using code inside debian/. Is there any reason
why you ruled out that option?

I also disagree with the need to go through NEW more than once. The new
package could quite simply be named libsystemd-private and lack a
.symbols and .shlibs file. Internal users would always use (=
${binary:Version}) anyway. The package description would declare that
any external dependency on libsystemd-private is a bug. Prior art:
libbinutils/binutils-dev.

One could argue that packaging a shared library like that is a policy
violation. If that is the case, then the current bundling is a policy
violation as well. So meh. It really sounds like a fair compromise to
me.

I am actually wondering now whether we should have a separate archive
section for "private" packages. I would define it as follows:

A private package is an implementation detail used by a single source.
Binary packages built from a one source package must not depend on
private packages built from another. Dependencies on private packages
should use tight version requirements (e.g. (= ${binary:Version}).

I suppose that a number of *-common and *-data packages as well as
gcc-N-base could be moved to such a section. User interfaces for package
managers could hide private packages by default.

Regardless of whether we add an archive section, lintian could carry a
list of private packages and enforce the no-dependency rule.

> Option 2 would be to drop Multi-Arch: foreign from systemd. This would mean,
> that packages like libpam-systemd/libnss-systemd can no longer be installed
> for a foreign architecture (even though those modules only use architecture
> independent interfaces of systemd).

That would break a pile of use cases and cause a lot of pain downstream.
Please don't.

> Option 3 is something I came up with after reading the Multi-Arch related
> wiki pages:
> https://salsa.debian.org/systemd-team/systemd/-/commit/c379461a14dcd13e0b625bd8faabbcf7fb3d19d6
> It would mean marking systemd as Multi-Arch: allowed. And packages that only
> use architecture interfaces of systemd would have to use the :any
> annotation.

Yeah, this technically works, but it causes a lot of maintenance effort,
because you get to hunt down every single systemd dependency in the
archive for years. I think our time is worth more than the cost of
introducing a new binary package into the archive.

> I would appreciate feedback, if option 3 is proper solution or not, or if
> there are other, better alternatives. If we go with option 3, should I
> inform all rdeps of systemd to update their dependency accordingly, i.e. do
> a MBF?

The problem I see here is less with annotating all deps once. It is more
the constant effort it incurs. It is not obvious that you should
annotate your systemd dependency :any. People will forget that when
adding systemd dependencies. What I take issue with is the permanent
cost that we keep in the long run.

Hope this helps

Helmut

Need help with Multi-Arch in systemd

[Michael Biebl]

Hi,

a couple of days ago, the following bug report was filed against systemd
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=990547

I'm quoting the relevant parts


I noticed that systemd-machined was failing to start on an arm64
box. This box had armhf enabled, and turns out systemd had been
cross-graded over to systemd:armhf[*]. It still had
systemd-container:arm64 installed, so now I had an arm64
/lib/systemd/systemd-machine, but an armhf
/lib/systemd/libsystemd-shared-244.so.

libsystemd-shared-244.so is from the systemd package. Since systemd is
marked Multi-Arch: foreign, systemd:armhf was able to incorrectly
satisfy systemd-container:arm64's dependency on systemd.


The systemd package provides a package private library
/lib/systemd/libsystemd-shared-{source:Version}.so, which is used by 
binaries that are built from src:systemd. Some of those binaries are 
split into separate binary packages, like systemd-container.
Since both systemd is marked as Multi-Arch: foreign, it can happen that 
we get this architecture mismatch.


Asking on #debian-systemd, Marco d'Itri suggested, that we move 
libsystemd-shared into a separate binary package. This would only help, 
if we moved libsystemd-shared into a Multi-Arch location (which means, 
we'd have to carry a patch against upstream). It would also mean, that 
on every new upstream release, systemd would have to go through NEW.

So I'm not very keen on doing that.

Option 2 would be to drop Multi-Arch: foreign from systemd. This would 
mean, that packages like libpam-systemd/libnss-systemd can no longer be 
installed for a foreign architecture (even though those modules only use 
architecture independent interfaces of systemd).


Option 3 is something I came up with after reading the Multi-Arch 
related wiki pages:

https://salsa.debian.org/systemd-team/systemd/-/commit/c379461a14dcd13e0b625bd8faabbcf7fb3d19d6
It would mean marking systemd as Multi-Arch: allowed. And packages that 
only use architecture interfaces of systemd would have to use the :any 
annotation.



I would appreciate feedback, if option 3 is proper solution or not, or 
if there are other, better alternatives. If we go with option 3, should 
I inform all rdeps of systemd to update their dependency accordingly, 
i.e. do a MBF?
Currently, I only see interpreters like perl, use M-A: allowed, so I'm 
not sure if I'm misusing this feature.


Regards,
Michael





OpenPGP_signature
Description: OpenPGP digital signature

Re: Help required to determine why some packages are being installed

[Dmitry Shachnev]

Hi David, and thanks for your reply!

On Thu, May 27, 2021 at 04:28:08PM +0200, David Kalnischkies wrote:
> The self-inflicted joy of avoiding a transitional package (see also the
> other thread about transitional packages on d-d@ I should comment) and
> of too strict dependencies (just because its the same source package
> doesn't mean = is a good choice; >= would have avoided that mess). ☺

Who could know that a removed package could make such a mess? :)

> Add the Breaks also to qtbase5-dev. Also, make that breaks versioned so
> that you can reintroduce qt5-default if that turns out to be needed
> (Lintian probably complains about it, too).

Thanks for the suggestion! Adding such Breaks solves most of the problems.
When I run “apt dist-upgrade”, apt no longer tries to remove libqt5gui5.

When I was testing this, I tried to upgrade only a subset of packages using
“apt install libqt5widgets5” (that package was installed, but running
that command should upgrade all Qt packages). Unfortunately, when running
this specific command, apt still wants to remove libqt5gui5 and install
libqt5gui5-gles. But I think we can live with it, as most users who have
not upgraded yet will probably use dist-upgrade (or full-upgrade).

> Usually I would recommend reintroducing qt5-default, but as you
> described it as a sid-user only problem it seems more beneficial to keep
> it removed for everyone rather than to readd for sid only confusing
> users who want to look up if a removal of qt5-default is okay or not.

To be more precise, it is a problem for users of sid and testing who have
not upgraded for a long time. Users of stable should be safe. Things are
a bit worse in Ubuntu, where the bad qt5-default made it to a stable
release. In any case, I discussed with my co-maintainer (Lisandro) and we
don't want to readd qt5-default.

--
Dmitry Shachnev

Re: Help required to determine why some packages are being installed

[David Kalnischkies]

Hi,

On Sun, May 16, 2021 at 10:00:29PM +0300, Dmitry Shachnev wrote:
> We determined that the reason for users getting libqt5gui5-gles installed
> is the qt5-default package. We removed it in October 2020 because it is no
> longer needed. But the latest version of that package that ever existed had
> this dependency:
> 
>   Depends: qtbase5-dev (= 5.14.2+dfsg-5) | qtbase5-gles-dev (>= 5.14.2+dfsg)
> 
> The latest available version of qtbase5-dev cannot satisfy that dependency,
> but the latest version of qtbase5-gles-dev can!

The self-inflicted joy of avoiding a transitional package (see also the
other thread about transitional packages on d-d@ I should comment) and
of too strict dependencies (just because its the same source package
doesn't mean = is a good choice; >= would have avoided that mess). ☺


> So for people who had qt5-default installed, apt tries to replace the normal
> Qt stack with -gles one to keep that dependency satisfied. It does so even
> if it's going to remove qt5-default anyway!

Yeah, greedy solver at its best. The problem is here mostly that an
earlier part of the solver has figured out (since bit more than a year
now) that the dependencies of qt5-default can't be satisfied (as it
tries to remove itself), but a) this knowledge isn't used much in the
caller yet and b) a later part who is usually responsible for contested
remove decisions isn't as good at figuring it out yet, so it decides to
try the bad gles subtree and as greedys are bad at reconsidering their
decisions it leads down to madness.

That specific case might actually be solvable on the apt side with some
more heavy work I intend to do eventually (just like I did for the
mentioned earlier part), but as you might expect, this isn't appropriate
to even think about while in freeze so don't hold your breath – and less
clear instances of this will probably remain out of reach still, so lets
look at our options:


> As an attempt to solve this problem, I added "Breaks: qt5-default" to both
> qtbase5-gles-dev and libqt5gui5-gles packages yesterday [2]. I thought this
> would convince apt to not consider these packages as a way to satisfy
> qt5-default dependency. But that did not work :(

Add the Breaks also to qtbase5-dev. Also, make that breaks versioned so
that you can reintroduce qt5-default if that turns out to be needed
(Lintian probably complains about it, too).

Usually I would recommend reintroducing qt5-default, but as you
described it as a sid-user only problem it seems more beneficial to keep
it removed for everyone rather than to readd for sid only confusing
users who want to look up if a removal of qt5-default is okay or not.


As qtbase5-dev is installed and depended on while nobody likes
qt5-default anymore the ProblemResolver will have an easy time figuring
out that removing qt5-default is better than holding back qtbase5-dev
would be and hence as a second step wont try to save qt5-default by
installing qtbase5-gles-dev (just to then figure out that gles-dev
breaks qt5-default, so it has to remove the later anyhow).


Thanks for looking into this: That seems like a simpler and less
controversial example than the one I used for the last big round of
resolver changes … and sorry it took me a while to reply.


Best regards

David Kalnischkies


signature.asc
Description: PGP signature

Re: Help required to determine why some packages are being installed

[Dmitry Shachnev]

Hi again David and all!

I want to ask for the help again.

We determined that the reason for users getting libqt5gui5-gles installed
is the qt5-default package. We removed it in October 2020 because it is no
longer needed. But the latest version of that package that ever existed had
this dependency:

  Depends: qtbase5-dev (= 5.14.2+dfsg-5) | qtbase5-gles-dev (>= 5.14.2+dfsg)

The latest available version of qtbase5-dev cannot satisfy that dependency,
but the latest version of qtbase5-gles-dev can!

So for people who had qt5-default installed, apt tries to replace the normal
Qt stack with -gles one to keep that dependency satisfied. It does so even
if it's going to remove qt5-default anyway!

As an attempt to solve this problem, I added "Breaks: qt5-default" to both
qtbase5-gles-dev and libqt5gui5-gles packages yesterday [2]. I thought this
would convince apt to not consider these packages as a way to satisfy
qt5-default dependency. But that did not work :(

I am attaching a log which was obtained as follows:

- Use sid snapshot from 2020-10-28 (a day before qt5-default was removed).
- Install qt5-default package.
- Change sources.list to use current sid.
- Upgrade apt to the latest version (just in case).
- Run "apt -o Debug::pkgProblemResolver=yes full-upgrade".

As can be seen in that log, apt still tries to remove libqt5gui5 and install
libqt5gui5-gles. If it did not do that, it would have to remove only one
package (qt5-default) and not three.

Can you please give me any advice on how to prevent apt from doing that?

[1]: 
https://tracker.debian.org/news/1187732/accepted-qtbase-opensource-src-5151dfsg-2-source-into-unstable/
[2]: 
https://tracker.debian.org/news/1241122/accepted-qtbase-opensource-src-gles-5152dfsg-4-source-into-unstable/

P.S. The suggestion from your last mail (see my quote below) did not work too.

On Sat, Jan 30, 2021 at 08:51:09PM +0300, Dmitry Shachnev wrote:
> Thanks for the suggestion! Having the reverse doesn’t hurt of course, so
> I have just added it in [1] and will include in the next upload. Let’s see
> if it makes things any better.
>
> [1]: https://salsa.debian.org/qt-kde-team/qt/qtbase/-/commit/4064bf01d808094d

--
Dmitry Shachnev
root@mitya57:/# apt -o Debug::pkgProblemResolver=yes full-upgrade
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Starting pkgProblemResolver with broken count: 1
Starting 2 pkgProblemResolver with broken count: 1
Investigating (0) qt5-default:amd64 < 5.14.2+dfsg-6 @ii mK Ib >
Broken qt5-default:amd64 Depends on qtbase5-dev:amd64 < 5.14.2+dfsg-6 -> 
5.15.2+dfsg-5 @ii umU > (= 5.14.2+dfsg-6)
  Considering qtbase5-dev:amd64 0 as a solution to qt5-default:amd64 -2
Broken qt5-default:amd64 Depends on qtbase5-gles-dev:amd64 < none | 
5.15.2+dfsg-4 @un uH > (>= 5.14.2+dfsg)
  Considering qtbase5-gles-dev:amd64 -1 as a solution to qt5-default:amd64 -2
  Try Installing qtbase5-gles-dev:amd64 < none | 5.15.2+dfsg-4 @un uH > before 
changing qt5-default:amd64
  Or group remove for qt5-default:amd64
Done
Calculating upgrade... Done
The following packages were automatically installed and are no longer required:
  fontconfig fontconfig-config fonts-dejavu-core libavahi-client3 
libavahi-common-data libavahi-common3 libboost-iostreams1.67.0 
libboost-iostreams1.71.0 libboost-system1.67.0 libbrotli1 libbsd0 libcups2 
libcwidget3v5 libdbus-1-3
  libdouble-conversion3 libdrm-amdgpu1 libdrm-common libdrm-intel1 
libdrm-nouveau2 libdrm-radeon1 libdrm2 libedit2 libegl-dev libegl-mesa0 libegl1 
libelf1 libevdev2 libexpat1 libfontconfig1 libfreetype6 libgbm1 libgl-dev libgl1
  libgl1-mesa-dri libglapi-mesa libgles2 libglib2.0-0 libglu1-mesa 
libglu1-mesa-dev libglvnd0 libglx-dev libglx-mesa0 libglx0 libgraphite2-3 
libgudev-1.0-0 libharfbuzz0b libice6 libicu67 libinput-bin libinput10 
libjpeg62-turbo libllvm11
  libmd0 libmd4c0 libmtdev1 libnss-nis libnss-nisplus libpciaccess0 
libpcre2-16-0 libperl5.30 libpng16-16 libpthread-stubs0-dev libqt5concurrent5 
libqt5core5a libqt5dbus5 libqt5gui5-gles libqt5network5 libqt5printsupport5 
libqt5sql5
  libqt5test5 libqt5widgets5 libqt5xml5 libsensors-config libsensors5 libsm6 
libvulkan-dev libvulkan1 libwacom-common libwacom2 libwayland-client0 
libwayland-server0 libx11-6 libx11-data libx11-dev libx11-xcb1 libxau-dev 
libxau6
  libxcb-dri2-0 libxcb-dri3-0 libxcb-glx0 libxcb-icccm4 libxcb-image0 
libxcb-keysyms1 libxcb-present0 libxcb-randr0 libxcb-render-util0 
libxcb-render0 libxcb-shape0 libxcb-shm0 libxcb-sync1 libxcb-util0 libxcb-util1 
libxcb-xfixes0
  libxcb-xinerama0 libxcb-xinput0 libxcb-xkb1 libxcb1 libxcb1-dev libxdamage1 
libxdmcp-dev libxdmcp6 libxext-dev libxext6 libxfixes3 libxkbcommon-x11-0 
libxkbcommon0 libxml2 libxrender1 libxshmfence1 libxxf86vm1 libz3-4
  perl-modules-5.30 qt5-qmake qt5-qmake-bin qtbase5-dev-tools qtchooser 
shared-mime-info ucf x11-common x11proto-core-dev x

Re: Need Help to build debian package from source code

[Nilesh Patra]

Hi,

On Thu, Mar 11, 2021 at 12:59:25PM +0530, Manikant Singh wrote:
> Hi Team,
> 
> I am IIT Kanpur student. I am trying to build debian package from source
> code after making some changes to it.
> Though I am able to make changes and build debian package successfully,
> some of the changes are not reflected. 
> 
> I made changes to corefile.c which are correctly reflected
> But changes made to histfile.c are not reflected after installation. 

It's really hard for any of us to help you unless you give details
of what you did, and what exactly do you aim to achieve.
Which package are you trying to build? Are you trying to make it into
official Debian? What changes did you make?

Please push your source package somewhere for a review

> I have followed this tutorial to create debian package.
> https://www.linuxfordevices.com/tutorials/debian/build-packages-from-source

Hmm, this looks like a hello world package. Why don't you follow our
wiki for the same[1], rather than other tutorials on the web?

There are also several wikis for setting up packaging environment et.
al. Do you have those things in place on your system?

[1]: https://wiki.debian.org/Packaging/Intro


> Could you help me with it ? Any help is appreciated. 

Details are necessary. But that being said, this list isn't the best place
for beginner packaging problems. Please use:
https://lists.debian.org/debian-mentors (debian-ment...@lists.debian.org) for 
these questions
(unless they are ambiguous or directly deal with policy matters or need the 
attention of several DDs)

> PS: I have tried with configure & make commands. installation works fine
> but I need debian package only.

Again, details please :-) Please put any follow up mails to
debian-mentors mailing list than here

Nilesh

Need Help to build debian package from source code

[Manikant Singh]

Hi Team,

I am IIT Kanpur student. I am trying to build debian package from source
code after making some changes to it.
Though I am able to make changes and build debian package successfully,
some of the changes are not reflected. 

I made changes to corefile.c which are correctly reflected
But changes made to histfile.c are not reflected after installation. 

I have followed this tutorial to create debian package.
https://www.linuxfordevices.com/tutorials/debian/build-packages-from-source

Could you help me with it ? Any help is appreciated. 

PS: I have tried with configure & make commands. installation works fine
but I need debian package only.

Manikant Singh
CSE M.Tech IITK

Bug#983842: ITP: srcode -- Tool that help developers to manage their codebase in an effective & productive way.

[Alois Micard]

Package: wnpp
Severity: wishlist
Owner: Aloïs Micard 

* Package name: srcode
  Version : 0.7.2-1
  Upstream Author : Aloïs Micard
* URL : https://github.com/creekorful/srcode
* License : GPL-3.0
  Programming Lang: Go
  Description : Tool that help developers to manage their codebase in an 
effective & productive way.

 srcode is a tool that help developers to manage their codebase in an effective
 & productive way.

 Cheers!

Re: Need help with getting a package to build reproducibly on arm*

[Vagrant Cascadian]

On 2021-01-08, Vagrant Cascadian wrote:
> On 2021-01-08, Vagrant Cascadian wrote:
>> On 2021-01-07, Vagrant Cascadian wrote:
>>> On 2021-01-07, Michael Biebl wrote:
>>>> Am 07.01.21 um 18:24 schrieb Michael Biebl:
>>>>> as can be seen at [1], systemd does not build reproducibly on armhf and
>>>>> arm64 (while there is no problem on amd64 and i386).
>>>>> 
>>>>> The problem is, I have no idea what the diffoscope diff [2] means and
>>>>> how I can make the package build reproducibly everywhere or how I can
>>>>> further investigate this.
>>>>> 
>>>>> Any help here is greatly appreciated as I think reproducible-builds are
>>>>> a great effort and I'd like to support that as much as I can.
>>>>> 
>>>>> Regards,
>>>>> Michael
>>>>> 
>>>>> 
>>>>> [1]
>>>>> https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/systemd.html
>>>>> [2]
>>>>> https://tests.reproducible-builds.org/debian/rb-pkg/unstable/arm64/diffoscope-results/systemd.html
>>>
>>> My best wild guesses would be parallelism, filesystem ordering or locale
>>> differences causing various sort ordering differences.
>>>
>>> I'm running a local build on arm64 with "reprotest --auto-build" to see
>>> if it can help give us any better leads, will see if that shows anything
>>> more useful... it could take some time on not particularly fast
>>> hardware.
>>
>> First attempt was reproducible for me (two normalized builds and one
>> varied build), though I couldn't vary the clock with reprotest
>> (libfaketime appears to trigger issues with building systemd)... or
>> fileordering, user, group or hostname due to some limitations my my
>> typical test environment. The command I ran was:
>>
>>   reprotest  --verbose --min-cpus=1 
>> --vary=-user_group,-domain_host,-fileordering,-time auto -- null
> ...
>
> But the second attempt for some reason did produce some interesting
> results... why it didn't happen the first time suggests it is not
> deterministic.
>
> │ │ │ ├── ./usr/bin/bootctl
> ...
> │ │ │ │ ├── strings --all --bytes=8 {}
> │ │ │ │ │ @@ -250,15 +250,15 @@
> │ │ │ │ │  SystemdOptions
> │ │ │ │ │  Failed to set SystemdOptions EFI variable: %m
> │ │ │ │ │  supported
> │ │ │ │ │  not supported
> │ │ │ │ │  Failed to query reboot-to-firmware state: %m
> │ │ │ │ │  Failed to parse argument: %s
> │ │ │ │ │  Failed to set reboot-to-firmware option: %m
> │ │ │ │ │ -/EFI/systemd/systemd-bootaa64.efi
> │ │ │ │ │ +/EFI/systemd/systemd-bootarm.efi
> │ │ │ │ │  Failed to access EFI variables. Is the "efivarfs" filesystem 
> mounted?
> │ │ │ │ │  Failed to determine current boot order: %m
>
> This suggests to me that the running kernel is somehow used to determine
> the userspace architecture, effectively similar to:
>
>   
> https://tests.reproducible-builds.org/debian/issues/unstable/captures_build_arch_issue.html
>
>
> The armhf builders on tests.reproducible-builds.org for Debian do not
> systematically test this. I'm not sure about the arm64 builders, but I
> think they run the same kernel, so it seems unlikely to be the only
> issue. Also surprised the i386 builder doesn't catch this. Then again,
> it only happened on my second try, which suggests this is
> non-deterministic in some way; maybe the slower armhf/aarch64
> architectures trigger it more often?
>
> I'll later post the results of the diffoscope output somewhere and give
> a closer look.

And those results I promised:

  https://people.debian.org/~vagrant/reproducible/systemd.20210108.dE8pOx/

Nothing terribly obvious to me, though as mentioned, the running kernel
may be a factor for the arm64 and armhf platforms.


live well,
  vagrant


signature.asc
Description: PGP signature

Re: Help required to determine why some packages are being installed

[Dmitry Shachnev]

Hi David, and thanks for your thorough reply and explanation!

On Sat, Jan 30, 2021 at 12:15:01PM +0100, David Kalnischkies wrote:
> To refine this: apt¹ picks the first alternative which is either:
> 1. already installed and satisfying as is
> 2. already installed, but needs an upgrade to satisfy
> 3. not installed, but can (may) be and would satisfy
>
> [...]

That was my understanding as well (and what you wrote below too).

> That said, I find it a bit odd that only libqt5gui5-gles conflicts with
> libqt5gui5. I doubt it will help apt, but it seems more honest to also
> have the reverse. Fun fact: having it only on one side actually gives
> the one having it a scoring advantage in apts conflict resolution, so
> for apt it reads in fact like -gles is the preferred package of the
> two making it less likely that apt holds back libqt5gui5. In practice
> other score points should level the playing field for libqt5gui5 though.
> (At least on my system more things depend on it than -gles provides).

Thanks for the suggestion! Having the reverse doesn’t hurt of course, so
I have just added it in [1] and will include in the next upload. Let’s see
if it makes things any better.

[1]: https://salsa.debian.org/qt-kde-team/qt/qtbase/-/commit/4064bf01d808094d

--
Dmitry Shachnev


signature.asc
Description: PGP signature

Re: Help required to determine why some packages are being installed

[David Kalnischkies]

On Wed, Jan 27, 2021 at 04:53:50PM +0100, Johannes Schauer Marin Rodrigues 
wrote:
> Quoting Jonas Smedegaard (2021-01-27 16:15:17)
> > I suspect that's not really the case - that instead apt tools might pick at
> > random.
> 
> no, apt does not pick at random. The apt solver prefers the first alternative.

To refine this: apt¹ picks the first alternative which is either:
1. already installed and satisfying as is
2. already installed, but needs an upgrade to satisfy
3. not installed, but can (may) be and would satisfy

(in reality you have to factor in explicit provides and implicit ones
 (like M-A:foreign) for all of them individual as well – and while apt
 has a partial order for those, you don't want to hear the details…
 just pretend its "clever" "random" between different providers of a
 given alternative).

(And for completeness "A|B, B|C" will have you end up with A and B
 installed except if e.g. A depends on C, then it will be A and C.
 So it's depth first for now. If you like this behaviour is a question
 of how preferable A is over B and how co-installable they are)

None of this is explicit requirement by Debian policy (which is silent
on many other things apt is required to do by tradition as well), it
only sort of follows out of "§7.1 Syntax" saying that any will do while
"§7.5 Virtual Packages" says that the default should be the first.


Note that if apt¹ can figure out that an alternative will not work it
will pick another alternative which may or may not be what you meant
with "is not available" as that (especially in unstable) includes if
a dependency of this alternative is not satisfied. Sounds innocent and
simple, right?

Consider that this includes all types of dependencies, not only the
positive depends on a not-yet build other package but also e.g. conflicts
with another package which must be part of the solution (= a direct
chain without other solutions from initial to this package exists) or
breaks you thought are there to force an upgrade of a package, but as
that version isn't built yet effectively means that the package has to
be removed instead.


So, without giving this a deep inspection my guess is simply that in
these cases apt managed to figure out that libqt5gui5 was not a good
alternative at the moment the user made the install/upgrade and went on
and choose the other. That's the point of alternatives after all.


In a way, that might be a regression of me working on those conundrums
last year to have apt figure out these things better to avoid breaking
recommends (in which these or-group and provides problems exist as well)
and letting apt find solutions which would previously be errors (as it
picked an unworkable alternative at first and wouldn't be able to
recover from it later on). So some (not all, apt always tried hard, it
just a bit more clever about it) of these users might previously be
greeted by apt error messages until the unstable archive stabilised
reporting bugs against apt as a solution "clearly" exists while apt will
now find that solution you would prefer it not to find it seems.


> I really would like to prevent this from happening for other users,
> so any suggestions would be welcome.

I don't think there is a solution to your problem – assuming that is
indeed this problem as I am only guessing – as at the very end its a
problem of the user accepting a solution they shouldn't, but to know
that you need to have specific domain knowledge. The hidden joys of
Debian unstable I guess… (doesn't help that it sort of sounds like
a package rename when libqt5gui5 is removed and -gles installed).


That said, I find it a bit odd that only libqt5gui5-gles conflicts with
libqt5gui5. I doubt it will help apt, but it seems more honest to also
have the reverse. Fun fact: having it only on one side actually gives
the one having it a scoring advantage in apts conflict resolution, so
for apt it reads in fact like -gles is the preferred package of the
two making it less likely that apt holds back libqt5gui5. In practice
other score points should level the playing field for libqt5gui5 though.
(At least on my system more things depend on it than -gles provides).


Best regards

David Kalnischkies


¹ Then I say apt here I mean the default resolver parts implemented in
libapt and used wholesale by pretty much everything using that library
including apt, apt-get, synaptics, various software centers, …
aptitude being a notable exception with its own resolver reusing only
some parts and reimplementing others which may or may not include the
parts responsible here (it is not an exaggeration when I say we have no
active Debian contributor who could answer that question as the only
person knowing how the aptitude resolver works went MIA years ago. So
the main reason it still (FSVO) works is that supercow is benevolent).
The other not

Re: Help required to determine why some packages are being installed

[Jonas Smedegaard]

Quoting Johannes Schauer Marin Rodrigues (2021-01-27 16:53:50)
> Quoting Jonas Smedegaard (2021-01-27 16:15:17)
> > I suspect that's not really the case - that instead apt tools might 
> > pick at random.
> 
> no, apt does not pick at random. The apt solver prefers the first 
> alternative.

With the plural "apt tools" I mean not only /usr/bin/apt but all tools 
linked with /usr/lib/*/libapt-pkg.so.* (and tools implementing same 
interfaces in other ways, if such exist).

Is there only one apt solver with only one possible configuration?

Maybe the word "random" is the wrong one to use, but do all those tools 
always pick the first option in all cases?

E.g. would they all _refuse_ to provide a solution that installed any 
packages if asked to install A, where A depends on B and C and B depends 
on "D1 or D2" and C depends on "D2 or D1"? - or which of those direct 
opposite "firsts" would in such scenario be deterministically picked?

It is my (vague!) understanding that the resolver used by aptitude (and, 
I assume, all apt tools) has priority between direct alternatives as one 
factor but other factors exist as well and the order and weight of 
factors is configurable to some degree.


> > It is my understanding that build daemons _ignore_ secondary entries 
> > exactly to avoid such ambiguity.
> 
> More precisely: not only build daemons but sbuild (even if you run it 
> locally) strips out all but the first alternative for Build-Depends, 
> Build-Depends-Indep and Build-Depends-Arch, with the exception of 
> those alternatives that name the same package as the first 
> alternative. Thus, this does *not* have an effect on alternatives in 
> binary packages.

Ah, I was unaware that _only_ build-dependencies was affected.  Thanks!


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

signature.asc
Description: signature

Re: Help required to determine why some packages are being installed

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Jonas Smedegaard (2021-01-27 16:15:17)
> I suspect that's not really the case - that instead apt tools might pick at
> random.

no, apt does not pick at random. The apt solver prefers the first alternative.

> It is my understanding that build daemons _ignore_ secondary entries
> exactly to avoid such ambiguity.

More precisely: not only build daemons but sbuild (even if you run it locally)
strips out all but the first alternative for Build-Depends, Build-Depends-Indep
and Build-Depends-Arch, with the exception of those alternatives that name the
same package as the first alternative. Thus, this does *not* have an effect on
alternatives in binary packages.

Thanks!

cheers, josch

signature.asc
Description: signature

Re: Help required to determine why some packages are being installed

[Jonas Smedegaard]

Quoting Lisandro Damián Nicanor Pérez Meyer (2021-01-27 15:09:41)
> So most packages now have an alternative dependency like libqt5gui5 
> (>= 5.x) | libqt5gui5-gles (>= 5.x). The strange thing is that *some* 
> users upgrading from stable or reinstalling packages [1] get the -gles 
> variant instead of the non-gles one by default. This causes bugs like 
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=976389
> 
> So far we couldn't reproduce the issue so we are kindly asking for
> your help in order to determine what's causing this behavior.

In above bugreport you write the following:

> All dependencies should be in a form like libqt5gui5 | 
> libqt5gui5-gles, so apt should fall back to the latter package only if 
> the former is not available for some reason. So I want to understand 
> what that reason is.

Is that really correct for all apt tools?  That the first is certain to 
be prioritized?

I suspect that's not really the case - that instead apt tools might pick 
at random.  It is my understanding that build daemons _ignore_ secondary 
entries exactly to avoid such ambiguity.


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

signature.asc
Description: signature

Help required to determine why some packages are being installed

[Lisandro Damián Nicanor Pérez Meyer]

Hi!

tl;dr: we the Qt maintainers need help to find out why some users
upgrading from stable / reinstalling packages get libqt5quick5-gles
instead of libqt5quick5.

Long story: some months ago my team mate Dmitry Shachnev managed to
get two flavours of Qt: the normal one with OpenGL support and the new
one with GLES support.

  https://mitya57.me/weblog/2020/01/qt-opengl-es-packages-available.html

  (If you care about the details I strongly recommend reading the
above blog post)

So most packages now have an alternative dependency like libqt5gui5
(>= 5.x) | libqt5gui5-gles (>= 5.x). The strange thing is that *some*
users upgrading from stable or reinstalling packages [1] get the -gles
variant instead of the non-gles one by default. This causes bugs like
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=976389

So far we couldn't reproduce the issue so we are kindly asking for
your help in order to determine what's causing this behavior.

Thanks in advance, Lisandro.

[1] There are some other cases in
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=976389#10

-- 
Lisandro Damián Nicanor Pérez Meyer
http://perezmeyer.com.ar/
http://perezmeyer.blogspot.com/

Re: Need help with getting a package to build reproducibly on arm*

[Michael Hudson-Doyle]

On Sat, 9 Jan 2021 at 00:52, Michael Biebl  wrote:

> Am 07.01.21 um 22:39 schrieb Michael Hudson-Doyle:
>
> > I'm hardly an expert in such things but it looks to me as if gcc is
> > ordering things differently between the two builds:
> >
> >
> https://tests.reproducible-builds.org/debian/dbd/unstable/arm64/systemd_247.2-4.diffoscope.html#systemd-tests_---.---_arm--.deb---data.tar.xz---data.tar---.-usr-lib-systemd-tests-manual-test-udev---objdump---line-numbers---disassemble---demangle---reloc---no-show-raw-insn---section-.text---
> >
> > and all the other differences follow on from that. Unless systemd is
> using
> > particularly strange compiler options,
>
> cflags:
>
> https://salsa.debian.org/systemd-team/systemd/-/blob/debian/master/meson.build#L343
>
> linkflags:
>
> https://salsa.debian.org/systemd-team/systemd/-/blob/debian/master/meson.build#L401


Those don't look particularly exotic to me.

We do build with LTO enabled.
>

I guess that could be the culprit. Would still be a gcc bug though AIUI.

Cheers,
mwh

Re: Need help with getting a package to build reproducibly on arm*

[Vagrant Cascadian]

On 2021-01-08, Vagrant Cascadian wrote:
> On 2021-01-07, Vagrant Cascadian wrote:
>> On 2021-01-07, Michael Biebl wrote:
>>> Am 07.01.21 um 18:24 schrieb Michael Biebl:
>>>> as can be seen at [1], systemd does not build reproducibly on armhf and
>>>> arm64 (while there is no problem on amd64 and i386).
>>>> 
>>>> The problem is, I have no idea what the diffoscope diff [2] means and
>>>> how I can make the package build reproducibly everywhere or how I can
>>>> further investigate this.
>>>> 
>>>> Any help here is greatly appreciated as I think reproducible-builds are
>>>> a great effort and I'd like to support that as much as I can.
>>>> 
>>>> Regards,
>>>> Michael
>>>> 
>>>> 
>>>> [1]
>>>> https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/systemd.html
>>>> [2]
>>>> https://tests.reproducible-builds.org/debian/rb-pkg/unstable/arm64/diffoscope-results/systemd.html
>>
>> My best wild guesses would be parallelism, filesystem ordering or locale
>> differences causing various sort ordering differences.
>>
>> I'm running a local build on arm64 with "reprotest --auto-build" to see
>> if it can help give us any better leads, will see if that shows anything
>> more useful... it could take some time on not particularly fast
>> hardware.
>
> First attempt was reproducible for me (two normalized builds and one
> varied build), though I couldn't vary the clock with reprotest
> (libfaketime appears to trigger issues with building systemd)... or
> fileordering, user, group or hostname due to some limitations my my
> typical test environment. The command I ran was:
>
>   reprotest  --verbose --min-cpus=1 
> --vary=-user_group,-domain_host,-fileordering,-time auto -- null
...

But the second attempt for some reason did produce some interesting
results... why it didn't happen the first time suggests it is not
deterministic.

│ │ │ ├── ./usr/bin/bootctl
...
│ │ │ │ ├── strings --all --bytes=8 {}
│ │ │ │ │ @@ -250,15 +250,15 @@
│ │ │ │ │  SystemdOptions
│ │ │ │ │  Failed to set SystemdOptions EFI variable: %m
│ │ │ │ │  supported
│ │ │ │ │  not supported
│ │ │ │ │  Failed to query reboot-to-firmware state: %m
│ │ │ │ │  Failed to parse argument: %s
│ │ │ │ │  Failed to set reboot-to-firmware option: %m
│ │ │ │ │ -/EFI/systemd/systemd-bootaa64.efi
│ │ │ │ │ +/EFI/systemd/systemd-bootarm.efi
│ │ │ │ │  Failed to access EFI variables. Is the "efivarfs" filesystem mounted?
│ │ │ │ │  Failed to determine current boot order: %m

This suggests to me that the running kernel is somehow used to determine
the userspace architecture, effectively similar to:

  
https://tests.reproducible-builds.org/debian/issues/unstable/captures_build_arch_issue.html


The armhf builders on tests.reproducible-builds.org for Debian do not
systematically test this. I'm not sure about the arm64 builders, but I
think they run the same kernel, so it seems unlikely to be the only
issue. Also surprised the i386 builder doesn't catch this. Then again,
it only happened on my second try, which suggests this is
non-deterministic in some way; maybe the slower armhf/aarch64
architectures trigger it more often?

I'll later post the results of the diffoscope output somewhere and give
a closer look.


live well,
  vagrant


signature.asc
Description: PGP signature

Bug#979587: ITP: ts-jest -- Node.js preprocessor with source maps support to help use TypeScript with Jest

[Xavier Guimard]

Package: wnpp
Severity: wishlist
Owner: Xavier Guimard 
X-Debbugs-Cc: debian-devel@lists.debian.org, 
pkg-javascript-de...@lists.alioth.debian.org

* Package name: ts-jest
  Version : 26.4.4
  Upstream Author : Kulshekhar Kabra <https://github.com/kulshekhar>
* URL : https://github.com/kulshekhar/ts-jest
* License : Expat
  Programming Lang: JavaScript
  Description : Node.js preprocessor with source maps support to help use 
TypeScript with Jest

Jest is a popular test framework for JavaScript projects. ts-jest
extends jest to test projects written in Typescript.

For now, some Debian packages keep untested due to the lack of this
package (for example, all node-dom* packages). It was not possible to
build ts-jest until now, due to lack of Jest typescript definitions
(fixed now).

ts-jest will be maintained under JS Team umbrella.

Re: Need help with getting a package to build reproducibly on arm*

[Michael Biebl]

Am 07.01.21 um 22:39 schrieb Michael Hudson-Doyle:


I'm hardly an expert in such things but it looks to me as if gcc is
ordering things differently between the two builds:

https://tests.reproducible-builds.org/debian/dbd/unstable/arm64/systemd_247.2-4.diffoscope.html#systemd-tests_---.---_arm--.deb---data.tar.xz---data.tar---.-usr-lib-systemd-tests-manual-test-udev---objdump---line-numbers---disassemble---demangle---reloc---no-show-raw-insn---section-.text---

and all the other differences follow on from that. Unless systemd is using
particularly strange compiler options,


cflags:
https://salsa.debian.org/systemd-team/systemd/-/blob/debian/master/meson.build#L343

linkflags:
https://salsa.debian.org/systemd-team/systemd/-/blob/debian/master/meson.build#L401

We do build with LTO enabled.




OpenPGP_signature
Description: OpenPGP digital signature

Re: Need help with getting a package to build reproducibly on arm*

[Vagrant Cascadian]

On 2021-01-07, Vagrant Cascadian wrote:
> On 2021-01-07, Michael Biebl wrote:
>> Am 07.01.21 um 18:24 schrieb Michael Biebl:
>>> as can be seen at [1], systemd does not build reproducibly on armhf and
>>> arm64 (while there is no problem on amd64 and i386).
>>> 
>>> The problem is, I have no idea what the diffoscope diff [2] means and
>>> how I can make the package build reproducibly everywhere or how I can
>>> further investigate this.
>>> 
>>> Any help here is greatly appreciated as I think reproducible-builds are
>>> a great effort and I'd like to support that as much as I can.
>>> 
>>> Regards,
>>> Michael
>>> 
>>> 
>>> [1]
>>> https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/systemd.html
>>> [2]
>>> https://tests.reproducible-builds.org/debian/rb-pkg/unstable/arm64/diffoscope-results/systemd.html
>
> My best wild guesses would be parallelism, filesystem ordering or locale
> differences causing various sort ordering differences.
>
> I'm running a local build on arm64 with "reprotest --auto-build" to see
> if it can help give us any better leads, will see if that shows anything
> more useful... it could take some time on not particularly fast
> hardware.

First attempt was reproducible for me (two normalized builds and one
varied build), though I couldn't vary the clock with reprotest
(libfaketime appears to trigger issues with building systemd)... or
fileordering, user, group or hostname due to some limitations my my
typical test environment. The command I ran was:

  reprotest  --verbose --min-cpus=1 
--vary=-user_group,-domain_host,-fileordering,-time auto -- null

So maybe one of those disabled variations, but all those are also varied
on all the platforms that tests.reproducible-builds.org tests for
Debian, so... hrm.


Another possibility is the locale used... reprotest picks more or less
at random, while:

  https://tests.reproducible-builds.org/debian/index_variations.html

  amd64: LANG="et_EE.UTF-8"
  i386: LANG="de_CH.UTF-8"
  arm64: LANG="nl_BE.UTF-8"
  armhf: LANG="it_CH.UTF-8"

Similar for LC_ALL and LANGUAGE.

But I somewhat doubt both nl_BE and it_CH would break in the same
way...


The other thing that's maybe a bit different is parallelism:

  XXX on amd64: 16 or 15
  XXX on i386: 10 or 9
  XXX on armhf: 5 or 3

But the difference between 3-5 cores and 9-10 or 15-16 doesn't seem very
likely to trigger issues either...


Was hoping reprotest would at least point us in a clearer direction for
what to test for ... but not today.

I'll chew on it a bit more and possibly try to stir up some more
possibilities.


live well,
  vagrant


signature.asc
Description: PGP signature

Re: Need help with getting a package to build reproducibly on arm*

[Vagrant Cascadian]

On 2021-01-07, Michael Biebl wrote:
> Am 07.01.21 um 18:24 schrieb Michael Biebl:
>> as can be seen at [1], systemd does not build reproducibly on armhf and
>> arm64 (while there is no problem on amd64 and i386).
>> 
>> The problem is, I have no idea what the diffoscope diff [2] means and
>> how I can make the package build reproducibly everywhere or how I can
>> further investigate this.
>> 
>> Any help here is greatly appreciated as I think reproducible-builds are
>> a great effort and I'd like to support that as much as I can.
>> 
>> Regards,
>> Michael
>> 
>> 
>> [1]
>> https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/systemd.html
>> [2]
>> https://tests.reproducible-builds.org/debian/rb-pkg/unstable/arm64/diffoscope-results/systemd.html

My best wild guesses would be parallelism, filesystem ordering or locale
differences causing various sort ordering differences.

I'm running a local build on arm64 with "reprotest --auto-build" to see
if it can help give us any better leads, will see if that shows anything
more useful... it could take some time on not particularly fast
hardware.


live well,
  vagrant


signature.asc
Description: PGP signature

Re: Need help with getting a package to build reproducibly on arm*

[Michael Hudson-Doyle]

On Fri, 8 Jan 2021 at 06:24, Michael Biebl  wrote:

> Hi,
>
> as can be seen at [1], systemd does not build reproducibly on armhf and
> arm64 (while there is no problem on amd64 and i386).
>

I'm hardly an expert in such things but it looks to me as if gcc is
ordering things differently between the two builds:

https://tests.reproducible-builds.org/debian/dbd/unstable/arm64/systemd_247.2-4.diffoscope.html#systemd-tests_---.---_arm--.deb---data.tar.xz---data.tar---.-usr-lib-systemd-tests-manual-test-udev---objdump---line-numbers---disassemble---demangle---reloc---no-show-raw-insn---section-.text---

and all the other differences follow on from that. Unless systemd is using
particularly strange compiler options, it's unlikely to be a systemd
packaging issue.

Cheers,
mwh


> The problem is, I have no idea what the diffoscope diff [2] means and
> how I can make the package build reproducibly everywhere or how I can
> further investigate this.
>
> Any help here is greatly appreciated as I think reproducible-builds are
> a great effort and I'd like to support that as much as I can.
>
> Regards,
> Michael
>
>
> [1]
>
> https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/systemd.html
> [2]
>
> https://tests.reproducible-builds.org/debian/rb-pkg/unstable/arm64/diffoscope-results/systemd.html
>
>

Re: Need help with getting a package to build reproducibly on arm*

[Michael Biebl]

[crossposting this to the reproducible-build mailing list. Please CC on 
replies, as I'm not subscribed]


Am 07.01.21 um 18:24 schrieb Michael Biebl:

Hi,

as can be seen at [1], systemd does not build reproducibly on armhf and
arm64 (while there is no problem on amd64 and i386).

The problem is, I have no idea what the diffoscope diff [2] means and
how I can make the package build reproducibly everywhere or how I can
further investigate this.

Any help here is greatly appreciated as I think reproducible-builds are
a great effort and I'd like to support that as much as I can.

Regards,
Michael


[1]
https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/systemd.html
[2]
https://tests.reproducible-builds.org/debian/rb-pkg/unstable/arm64/diffoscope-results/systemd.html

Need help with getting a package to build reproducibly on arm*

[Michael Biebl]

Hi,

as can be seen at [1], systemd does not build reproducibly on armhf and 
arm64 (while there is no problem on amd64 and i386).


The problem is, I have no idea what the diffoscope diff [2] means and 
how I can make the package build reproducibly everywhere or how I can 
further investigate this.


Any help here is greatly appreciated as I think reproducible-builds are 
a great effort and I'd like to support that as much as I can.


Regards,
Michael


[1] 
https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/systemd.html
[2] 
https://tests.reproducible-builds.org/debian/rb-pkg/unstable/arm64/diffoscope-results/systemd.html

Re: Help with contacting Ubuntu devs for updating critically broken packages

[Eli Schwartz]

Thank you, Norbert, for trying to get this fixed. calibre upstream
already had 6 bug reports and counting in just the last week, for an issue:

- that was fixed in May
- only applicable to people field-testing the python3 transition which
  was marked as beta and is currently superseded by stable releases

Obviously, upstream getting bug reports due to distro respins shipping
*outdated* and broken beta code is one of those "non-ideal" situations...

> The upload needs to be confirmed that it fixes the problem by someone
> else than you before it gets into Ubuntu proper. I haven't had any good
> experience with it.

I guess it would be nice if "I'm the debian maintainer, you accidentally
pulled a beta build which is RC-buggy" had some weight to it.

We'll see how it goes, I suppose. There will be major problems if Ubuntu
keeps this in an LTS release for the next 5 years.

-- 
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature