Re: Tainted builds (was Re: usrmerge -- plan B?)

[Russ Allbery]

Guillem Jover  writes:

> Whether a package is being built within a chroot or not, has nothing
> to do with how that installation is being managed IMO. It feels a bit
> like recording what's the form factor of the machine being run on? :)

I think what people are trying to get at here is "was the package built on
a system with packages other than build dependencies plus build-essential
plus essential/required packages installed."

I do think this would be very useful to track, but it's a bit complicated
to work out, and there are probably a few other exceptions that would need
to be in place.

-- 
Russ Allbery (r...@debian.org)   

Packaging runscripts for Runit init system

[Dmitry Bogatov]

[ Forgot to mention. Please keep me in CC. ]

> [ Tollef Fog Heen ]
>  For the record, the TC expects maintainers to continue to support the
>  multiple available init systems in Debian.  That includes merging
>  reasonable contributions, and not reverting existing support without
>  a compelling reason.

I did not remember this quote from TC. Sounds like solution.

Not that I think it is the best way, but since Debian already provide
sysvinit and systemd scripts as part of main package, I will follow this
tradition. Probably, we could add several sentences into policy,
deprecating *-run packages.

Idea with creating logging users with trigger on `runit' installation
also sounds nice. I will definitely take a look.

Lorenzo, another bright side is that it solves ownership question. There
will be no question, what to do if at time of -run package installation
daemon already running.

Users of runit-sysv/runit-systemd (which I always considered as
non-priority) will have to manually choose, who of sysvinit/runit/systemd
will manage given daemon.


pgpNwXpbXAzqo.pgp
Description: PGP signature

Bug#915358: ITP: magic-wormhole-mailbox-server -- Magic Wormhole Mailbox Server

[Antoine Beaupre]

Package: wnpp
Severity: wishlist
Owner: Antoine Beaupre 

* Package name: magic-wormhole-mailbox-server
  Version : 0.3.1
  Upstream Author : Brian Warner
* URL : https://github.com/warner/magic-wormhole-mailbox-server
* License : MIT
  Programming Lang: Python
  Description : Magic Wormhole Mailbox Server

This is the main server that Magic-Wormhole clients connect to. The
server performs store-and-forward delivery for small key-exchange and
control messages. Bulk data is sent over a direct TCP connection, or
through a transit-relay.

Clients connect with WebSockets, for low-latency delivery in the happy
case where both clients are attached at the same time. Message are
stored to enable non-simultaneous clients to make forward
progress. The server uses a small SQLite database for persistence (and
clients will reconnect automatically, allowing the server to be
rebooted without losing state). An optional "usage DB" tracks
historical activity for status monitoring and operational maintenance.

==

This is necessary for magic-wormhole 0.11.2 tests to pass as they
include code from this package, which is now split out in a
different upstream repository.

It can also be useful for people who want to run their own custom
wormhole-based system without using any of the global wormhole
architecture.

Re: Tainted builds (was Re: usrmerge -- plan B?)

[Guillem Jover]

On Wed, 2018-11-28 at 14:48:32 -0200, Antonio Terceiro wrote:
> On Wed, Nov 28, 2018 at 02:57:52PM +0100, Guillem Jover wrote:
> > This is actually a great idea! I went ahead and implemented this, see
> > attached tentative patch which I'm planning on including in dpkg 1.19.3.
> 
> Would you be willing to also implement
> 
>   Tainted-By: not-built-in-a-chroot
> 
> ?

I also agree with what others have mentioned down-thread, that this is
not really a tainting reason. Tainting would involve say, programs
or libraries under /usr/local/{bin,sbin/lib}/, perhaps even configs
under /usr/local/etc, or even /etc with conffiles diverged from the
defaults, say output from «dpkg --verify», files unowned by a package
(in the sense of either being tracked in dpkg database or whether the
package manually manages the pathnames) being present in directories
in the executable or linker search paths, etc.

While some of these might be reliable, easy to emit and cheap to
compute, doing for example «dpkg --verify» before every build would
be IMO unnacceptably slow, but would be a definitive answer to one
kind of tainted system.

Whether a package is being built within a chroot or not, has nothing
to do with how that installation is being managed IMO. It feels a bit
like recording what's the form factor of the machine being run on? :)

> a few nits in the manpage:

Great! Thanks for the review, I've tried to reword and improve several
of the docs, attached updated patch.

Thanks,
Guillem
From ca2b0f9710867f3d8e8bca3a6654b9dd67db3539 Mon Sep 17 00:00:00 2001
From: Guillem Jover 
Date: Sun, 2 Dec 2018 03:35:49 +0100
Subject: [PATCH] dpkg-genbuildinfo: Include a new Build-Tainted-By field

This field will contain a list of tainting reason tags, which can denote
that the current build has potentially been broken.

On Debian and derivatives it can emit now merged-usr-via-symlinks as a
tainting reason tag.

Suggested-by: Alexander E. Patrakov 
---
 man/deb-buildinfo.man  | 19 +++
 scripts/Dpkg/Control/FieldsCore.pm |  7 ++-
 scripts/Dpkg/Vendor/Debian.pm  | 20 
 scripts/Dpkg/Vendor/Default.pm | 10 ++
 scripts/dpkg-genbuildinfo.pl   |  2 ++
 5 files changed, 57 insertions(+), 1 deletion(-)

diff --git a/man/deb-buildinfo.man b/man/deb-buildinfo.man
index 5013aa047..0708fa319 100644
--- a/man/deb-buildinfo.man
+++ b/man/deb-buildinfo.man
@@ -149,6 +149,25 @@ via some pattern match to avoid leaking possibly sensitive information.
 On Debian and derivatives only build paths starting with \fI/build/\fP
 will emit this field.
 .TP
+.B Build\-Tainted\-By:
+.TQ
+.I " taint-reason-list"
+This folded field contains a space-separated list of reason tags (formed by
+alphanumeric and dash characters) which identify why the current build has
+been tainted (since dpkg 1.19.3).
+.IP
+On Debian and derivatives the following reason tags can be emitted:
+.RS
+.TP
+.B merged\-usr\-via\-symlinks
+The system has a merged \fI/usr\fP via symlinks.
+This will confuse \fBdpkg\-query\fP as it messes with the understanding
+of the filesystem that \fBdpkg\fP has recorded in its database.
+For build systems that hardcode pathnames to specific binaries or libraries
+on the resulting artifacts, it can also produce packages that will be
+incompatible with non-/usr-merged filesystems.
+.RE
+.TP
 .BR Installed\-Build\-Depends: " (required)"
 .TQ
 .I " package-list"
diff --git a/scripts/Dpkg/Control/FieldsCore.pm b/scripts/Dpkg/Control/FieldsCore.pm
index b100366e1..f460433fc 100644
--- a/scripts/Dpkg/Control/FieldsCore.pm
+++ b/scripts/Dpkg/Control/FieldsCore.pm
@@ -176,6 +176,11 @@ our %FIELDS = (
 allowed => CTRL_INFO_PKG,
 separator => FIELD_SEP_SPACE,
 },
+'build-tainted-by' => {
+name => 'Build-Tainted-By',
+allowed => CTRL_FILE_BUILDINFO,
+separator => FIELD_SEP_SPACE,
+},
 'built-for-profiles' => {
 name => 'Built-For-Profiles',
 allowed => ALL_PKG | CTRL_FILE_CHANGES,
@@ -634,7 +639,7 @@ our %FIELD_ORDER = (
 qw(format source binary architecture version binary-only-changes),
 @src_checksums_fields,
 qw(build-origin build-architecture build-kernel-version build-date
-build-path installed-build-depends environment),
+build-path build-tainted-by installed-build-depends environment),
 ],
 CTRL_FILE_CHANGES() => [
 qw(format date source binary binary-only built-for-profiles architecture
diff --git a/scripts/Dpkg/Vendor/Debian.pm b/scripts/Dpkg/Vendor/Debian.pm
index 7d4b6d802..6948bdc16 100644
--- a/scripts/Dpkg/Vendor/Debian.pm
+++ b/scripts/Dpkg/Vendor/Debian.pm
@@ -81,6 +81,8 @@ sub run_hook {
 $self->_add_build_flags(@params);
 } elsif ($hook eq 'builtin-system-build-paths') {
 return qw(/build/);
+} elsif ($hook eq 'build-tainted-by') {
+return $self->_build_tainted_by();
 } else {
 return $self->SUPER::run_hook($ho

Re: opa-ff patch-queue

[Guido Günther]

Hi,
On Fri, Nov 30, 2018 at 06:37:54PM -0600, Brian Smith wrote:
> Greetings,
> 
> I've been looking into updating opa-ff to the upstream 10.8.0.0.201
> release and have some questions about the patch-queue process
> specified in d/README.source, which references DEP-14.
> 
> The document states that the upstream tag should be merged to the
> patch-queue/debian/master branch. However, after doing that and
> executing gbp pq export, it generates a patch that upgrades the source
> to the latest version, since gbp-pq sees the merge commit.
> 
> The process I've found that "works" is:
> 1) Merge the upstream tag to debian/master.
> 2) Refresh the existing patches and fix any conflicts.
> 3) Execute gbp pq switch.
> 
> At this point, debian/master is merged to patch-queue/debian/master.
> patch-queue/debian/master now contains d/patches and each patch has to
> be applied and the changes committed to git.
> 
> Also, it appears that the patch-queue/debian/master branch was dropped
> and recreated by the gbp-pq export command, as the previous commits to
> that branch are now removed.
> 
> I stopped here, as this didn't really feel quite right. There doesn't
> appear to be a lot of supporting documentation for the gbp pq
> workflow.

http://honk.sigxcpu.org/projects/git-buildpackage/manual-html/gbp.patches.html

 -- Guido

> 
> Can someone point me to a document or clarify how this process is
> supposed to work, regarding the rebase of the patch-queue branch to
> the latest release and exporting the updated patches?
> 
> -- 
> Brian T. Smith
> System Fabric Works
> Senior Technical Staff
> bsm...@systemfabricworks.com
> GPG Key: 0xB3C2C7B73BA3CD7F
> 

Re: wicd-daemon-run_1.0_amd64.changes REJECTED

[Guillem Jover]

On Thu, 2018-11-29 at 13:56:40 -0800, Josh Triplett wrote:
> Preferably in a package maintained by someone who actually uses that
> daemon with sysvinit, rather than one maintained by someone who doesn't.
> (And bugs in the use of that package with sysvinit then belong to that
> separate package, where the work can be done by people who want that
> functionality. Also, s/All packages/All packages capable of running with
> sysvinit/.)

I very much disagree over this as a general principle, be it for
sysvinit or systemd, or whatever other feature a package might provide
that might be optionally used, but the maintainer does not use.

This is the antithesis of what a distribution is supposed to be doing.

> Small packages are not a bug. Our archive not handling them well (to the
> extent it doesn't, which seems questionable) is a bug. And it's
> absolutely a feature to have runit scripts and sysvinit scripts and
> configuration snippets of many sorts maintained in separate packages.

And I still object to this characterization, because the metadata is
the lesser of the problems when it comes to overhead. Few small packages
might be fine, tons of them is an organizational bug:

  

Thanks,
Guillem

Re: CTTE decision on vendor-specific patch series (bug #904302)

[Guillem Jover]

On Tue, 2018-11-13 at 19:22:09 +0100, Tollef Fog Heen wrote:
>  Resolution 
[…]
> The Committee therefore resolves that:
> 
> 1. Any use of dpkg's vendor-specific patch series feature is a bug for
>packages in the Debian archive (including contrib and non-free).
> 
>This should be implemented in Debian Policy by declaring that a
>package SHOULD NOT contain a non-default series file.
> 
> 2. After Buster is released, use of the vendor-specific patch series
>feature is forbidden in the Debian archive.
> 
>This should be implemented in Debian Policy by declaring that a
>package MUST NOT contain a non-default series file.
> 
>  End of resolution 

As I already mentioned in debian-derivatives
:

  ,---
  And to make this crystal clear, I have no plan whatsoever to remove
  this feature from dpkg itself, so derivatives will still be able to
  use it for themselves and their own derivatives going forward. I'll
  probably even improve upon it, so that even more code can be turned
  into declarative files instead of ad-hoc gook all over the place.
  `---

For example starting with 1.19.2, the series file pathname used is
printed now when applying patches, and it was properly documented.

Regards,
Guillem

Bug#915288: ITP: erlang-p1-pkix -- PKIX certificates management library for Erlang

[Philipp Huebner]

Package: wnpp
Severity: wishlist
Owner: Philipp Huebner 

* Package name: erlang-p1-pkix
  Version : 0.2018.11.12
  Upstream Author : ProcessOne
* URL : https://github.com/processone/pkix
* License : Apache-2.0
  Programming Lang: Erlang
  Description : PKIX certificates management library for Erlang

 The idea of the library is to simplify certificates configuration in Erlang
 programs. Typically an Erlang program which needs certificates  (for HTTPS/
 MQTT/XMPP/etc) provides a bunch of options such as certfile,  chainfile,
 privkey, etc. The situation becomes even more complicated when a  server
 supports so called virtual domains because a program is typically  required to
 match a virtual domain with its certificate. If a user has plenty  of virtual
 domains it's quickly becoming a nightmare for them to configure all this.
 The complexity also leads to errors: a single configuration mistake and a
 program generates obscure log messages, unreadable Erlang tracebacks or,
 even worse, just silently ignores the errors.
 Fortunately, the large part of certificates configuration can be automated,
 reducing a user configuration to something as simple as:
 .
 certfiles:
   - /etc/letsencrypt/live/*/*.pem
 .
 The purpose of this library is to do this dirty job under the hood.
 
 This package is needed for future ejabberd releases.
 VCS at https://salsa.debian.org/ejabberd-packaging-team/erlang-p1-pkix

Re: usrmerge -- plan B?

[Niko Tyni]

On Fri, Nov 23, 2018 at 04:32:17PM +0100, Guillem Jover wrote:

> Please, if we decide we want to do merged /usr, let's do it properly.

I'm toying with the idea of creating a merged-usr package indicating
'this system has merged /usr'. It could either fail to install with an
informational message on unmerged-usr systems, or if we're brave enough,
interactively or automatically convert the system (possibly utilizing
the usrmerge package.)

Then debhelper (or even dpkg-dev) could probe whether the build system
has merged /usr, and automatically add dependencies on merged-usr to
resulting binary packages, The dependency injection could be on either
an opt-out or an opt-in basis.

The opt-out way would be the more safe and conservative approach,
assuming everything built on merged /usr is broken on non-merged /usr
unless the maintainer indicates otherwise.

On the contrary, the opt-in approach would require that all problematic
packages (those that will not function fine when built on merged /usr
systems but installed on non-merged /usr ones) be identified and tagged
as needing this dependency. This obviously has a chance of missing
some of the issues.

Whether opt-out or opt-in, the transition in Debian could then be
controlled by monitoring dependencies on merged-usr, possibly playing
whack-a-mole rebuilding things that have gained this dependency in
maintainer uploads, or letting the gates open at some point and allowing
even core packages to gain this dependency (effectively forcing /usr
merge on all systems.)

All this would make the issues explicit and reflected in the package
dependency metadata, which seems to me like at least a step forward.
-- 
Niko Tyni   nt...@debian.org

Bug#915265: ITP: o2 -- implementation of a communication protocol for music systems

[IOhannes m zmoelnig]

Package: wnpp
Severity: wishlist
Owner: IOhannes m zmoelnig 

* Package name: o2
  Version : 1.0
  Upstream Author : Roger Dannenberg
* URL : https://rbdannenberg.github.io/o2/
* License : MIT/X
  Programming Lang: C
  Description : implementation of a communication protocol for music systems

 O2 is a communication protocol for interactive music and media applications.
 It is inspired by Open Sound Control (OSC) and uses similar means to form
 addresses, specify types, and encode messages.
 .
 In addition to providing message delivery, O2 offers a discovery mechanism
 where processes automatically discover and connect to other processes.
 Furthermore, O2 implements a clock synchronization protocol.
 .
 O2 is based on IP (Internet Protocol), but there are some mechanisms that allow
 an O2 process to serve as a bridge to other networks such as Bluetooth.

I intend to maintain this under the multimedia-team umbrella.

AMD64 problem: mips cross gcc munmap_chunk(): invalid pointer (on AVX cpu only)

[Yunqiang Su]

I meet a problem of Debian’s cross toolchain.[1]
It happens for gcc-7/gcc-8 in Debian Sid/Buster, while not for gcc-6 in stretch.

This problem only appears on amd64, not on i386.
And this problems seems only on CPUs with AVX support, I have an old HP server, 
with CPU
   Intel(R) Xeon(R) CPU E7- 4807  @ 1.87GHz
works well. Which don’t support AVX.

Anybody has machines with AMD CPUs?

FYI: Windows Subsystem for Linux has this problem, too.

1. Install Debian testing/sid env on amd64
2. apt-get install gcc-8-mips64el-linux-gnuabi64 gcc-8-mips64-linux-gnuabi64 
gcc-8-mips-linux-gnu

These bellow cmd will show:
 munmap_chunk(): invalid pointer
 Aborted

$ echo "int a(){ return 1; }" | mips64el-linux-gnuabi64-gcc-8 -c -mabi=32 -xc -
$ echo "int a(){ return 1; }" | mips64el-linux-gnuabi64-gcc-8 -B/non_exists/ -c 
-mabi=32 -xc -
$ echo "int a(){ return 1; }" | mips64el-linux-gnuabi64-gcc-8 -B/usr/share -c 
-mabi=32 -xc -
$ echo "int a(){ return 1; }" | mips64el-linux-gnuabi64-gcc-8 -EB -c -mabi=32 
-xc -


These bellow cmd works well
# for both n32 or 64 abi, it works well
$ echo "int a(){ return 1; }" | mips64el-linux-gnuabi64-gcc-8 -c -mabi=n32 -xc -
$ echo "int a(){ return 1; }" | mips64el-linux-gnuabi64-gcc-8 -c -mabi=64 -xc -

# -B a path which contains MIPS cross assembler
$ echo "int a(){ return 1; }" | mips64el-linux-gnuabi64-gcc-8 
-B/usr/mips64el-linux-gnuabi64/bin/ -c -mabi=32 -xc -
$ echo "int a(){ return 1; }" | mips64el-linux-gnuabi64-gcc-8 -B/usr/bin/ -c 
-mabi=32 -xc -

# none mips64el-linux-gnuabi64 toolchains works well.
$ echo "int a(){ return 1; }" | mips64-linux-gnuabi64-gcc-8 -c -mabi=32 -xc -
$ echo "int a(){ return 1; }" | mips64-linux-gnuabi64-gcc-8  -EL -c -mabi=32 
-xc -
$ echo "int a(){ return 1; }" | mipsel-linux-gnu-gcc-8 -c -mabi=64 -xc -
$ echo "int a(){ return 1; }" | mipsel-linux-gnu-gcc-8 -c -mabi=n32 -xc -
$ echo "int a(){ return 1; }" | mipsel-linux-gnu-gcc-8 -c -mabi=32 -xc -

With some debug, this problem is triggered in  gcc/gcc.c
static int
execute (void)
{
 ….
  pex_free (pex);


[1]  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=915194
[2] gcc -v for these toolchains.

xxx@xxx:~$ mipsel-linux-gnu-gcc-8 -v
Using built-in specs.
COLLECT_GCC=mipsel-linux-gnu-gcc-8
COLLECT_LTO_WRAPPER=/usr/lib/gcc-cross/mipsel-linux-gnu/8/lto-wrapper
Target: mipsel-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Debian 8.2.0-10' 
--with-bugurl=file:///usr/share/doc/gcc-8/README.Bugs 
--enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++ --prefix=/usr 
--with-gcc-major-version-only --program-suffix=-8 --enable-shared 
--enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext 
--enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ 
--enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes 
--with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-libitm 
--disable-libsanitizer --disable-libquadmath --disable-libquadmath-support 
--enable-plugin --enable-default-pie --with-system-zlib --disable-libphobos 
--enable-multiarch --disable-werror --enable-multilib --with-arch-32=mips32r2 
--with-fp-32=xx --with-madd4=no --with-lxc1-sxc1=no --enable-targets=all 
--with-arch-64=mips64r2 --enable-checking=release --build=x86_64-linux-gnu 
--host=x86_64-linux-gnu --target=mipsel-linux-gnu 
--program-prefix=mipsel-linux-gnu- --includedir=/usr/mipsel-linux-gnu/include
Thread model: posix
gcc version 8.2.0 (Debian 8.2.0-10)

xxx@xxx:~$ mips64el-linux-gnuabi64-gcc-8 -v
Using built-in specs.
COLLECT_GCC=mips64el-linux-gnuabi64-gcc-8
COLLECT_LTO_WRAPPER=/usr/lib/gcc-cross/mips64el-linux-gnuabi64/8/lto-wrapper
Target: mips64el-linux-gnuabi64
Configured with: ../src/configure -v --with-pkgversion='Debian 8.2.0-10' 
--with-bugurl=file:///usr/share/doc/gcc-8/README.Bugs 
--enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++ --prefix=/usr 
--with-gcc-major-version-only --program-suffix=-8 --enable-shared 
--enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext 
--enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ 
--enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes 
--with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-libitm 
--disable-libsanitizer --disable-libquadmath --disable-libquadmath-support 
--enable-plugin --enable-default-pie --with-system-zlib --disable-libphobos 
--enable-multiarch --disable-werror --enable-multilib --with-mips-plt 
--with-arch-64=mips64r2 --with-madd4=no --enable-targets=all 
--with-arch-32=mips32r2 --with-fp-32=xx --enable-checking=release 
--build=x86_64-linux-gnu --host=x86_64-linux-gnu 
--target=mips64el-linux-gnuabi64 --program-prefix=mips64el-linux-gnuabi64- 
--includedir=/usr/mips64el-linux-gnuabi64/include
Thread model: posix
gcc version 8.2.0 (Debian 8.2.0-10)

xxx@xxx:~$  mips64-linux-gnuabi64-gcc-8 -v
Using built-in specs.
COLLECT_GCC=mips64-linux-gnuabi64-gcc-8
COLLECT_LTO_WRAPPER=/usr/lib/gcc-cross/mips64-linux-gn