Bug#1087026: ITP: golang-github-tillitis-tkeysign -- Go package for communicating with Tillitis TKey signer

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 

* Package name: golang-github-tillitis-tkeysign
  Version : 1.0.1-1
  Upstream Author : Tillitis AB
* URL : https://github.com/tillitis/tkeysign
* License : BSD-2-clause
  Programming Lang: Go
  Description : Go package for communicating with Tillitis TKey signer

 Tillitis TKey Sign package
 .
 A Go package for communicating with the signer device app
 (https://github.com/tillitis/tkey-device-signer) on a Tillitis
 (https://tillitis.se/) TKey to get cryptographic signatures over a
 message.
 .
 See the Go doc (https://pkg.go.dev/github.com/tillitis/tkeysign) for
 tkeysign for details on how to call the functions.
 .
 See tkey-ssh-agent (https://github.com/tillitis/tkey-ssh-agent) and tkey-sign-
 cli (https://github.com/tillitis/tkey-sign-cli) for client applications
 using this go package.

I hope to maintain this package as part of Debian Go Packaging Team:

https://salsa.debian.org/go-team/packages/golang-github-tillitis-tkeysign

/Simon


signature.asc
Description: PGP signature

Bug#1087025: ITP: golang-github-tillitis-tkeyutil -- A Go package with utility functions for a client application communicating with a Tillitis TKey.

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 

* Package name: golang-github-tillitis-tkeyutil
  Version : 0.0.8-1
  Upstream Author : Tillitis AB
* URL : https://github.com/tillitis/tkeyutil
* License : BSD-2-clause
  Programming Lang: Go
  Description : A Go package with utility functions for a client 
application communicating with a Tillitis TKey.

 Tillitis TKey Utility package
 .
 A Go package with utility functions for a client application
 communicating with  a Tillitis (https://tillitis.se/) TKey.
 .
 See the Go doc (https://pkg.go.dev/github.com/tillitis/tkeyutil) for
 tkeyutil for details on how to call the functions.
 .
 See tkey-ssh-agent (https://github.com/tillitis/tkey-ssh-agent/) for an
 example client application.

I hope to maintain this package as part of Debian Go Packaging Team:

https://salsa.debian.org/go-team/packages/golang-github-tillitis-tkeyutil

/Simon


signature.asc
Description: PGP signature

Bug#1087024: ITP: golang-github-tillitis-tkeyclient -- Go package for a TKey client

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 

* Package name: golang-github-tillitis-tkeyclient
  Version : 1.1.0-1
  Upstream Author : Tillitis AB
* URL : https://github.com/tillitis/tkeyclient
* License : BSD-2-clause
  Programming Lang: Go
  Description : Go package for a TKey client

 Tillitis TKey Client package
 .
 A Go package for controlling a Tillitis (https://tillitis.se/) TKey,
 upload device apps, and communicate with it.
 .
 See the Go doc (https://pkg.go.dev/github.com/tillitis/tkeyclient) for
 tkeyclient for details on how to call the functions.
 .
 See tkey-ssh-agent (https://github.com/tillitis/tkey-ssh-agent) or tkeysign
 (https://github.com/tillitis/tkeysign) for example applications using
 this Go package.

I hope to maintain this package as part of Debian Go Packaging Team:

https://salsa.debian.org/go-team/packages/golang-github-tillitis-tkeyclient

/Simon


signature.asc
Description: PGP signature

Bug#1087023: ITP: tkey-ssh-agent -- SSH Agent for TKey, the flexible open hardware/software USB security key 🔑

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 

* Package name: tkey-ssh-agent
  Version : 1.0.0-1
  Upstream Author : Tillitis AB
* URL : https://github.com/tillitis/tkey-ssh-agent
* License : GPL-2.0
  Programming Lang: Go
  Description : SSH Agent for TKey, the flexible open hardware/software USB 
security key 🔑

 TKey SSH Agent
 .
 tkey-ssh-agent is an OpenSSH-compatible agent for use with the Tillitis
 (https://tillitis.se/) TKey USB security token.

I hope to maintain this package as part of Debian Go Packaging Team:

https://salsa.debian.org/go-team/packages/tkey-ssh-agent

/Simon


signature.asc
Description: PGP signature

Bug#1086139: ITP: golang-github-c2sp-cctv -- Community Cryptography Test Vectors

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 

* Package name: golang-github-c2sp-cctv
  Version : 0.0~git20240306.3ec4d71-1
  Upstream Author : Community Cryptography Specification Project
* URL : https://github.com/C2SP/CCTV
* License : https://github.com/C2SP/CCTV/issues/14
  Programming Lang: Go
  Description : Community Cryptography Test Vectors

 The Community Cryptography Test Vectors
 .
 CCTV is an experiment, part of the Community Cryptography Specification
 Project (https://c2sp.org), in test vector collection and reuse.
 .
 All cryptography-related test vectors are welcome, and projects are
 encouraged to reuse them and contribute back any new vectors they
 generate.

I hope to maintain this package as part of Debian Go Packaging Team:

https://salsa.debian.org/go-team/packages/golang-github-c2sp-cctv

/Simon


signature.asc
Description: PGP signature

Bug#1085874: ITP: golang-github-theupdateframework-go-tuf-v2 -- The Update Framework (TUF) v2 Go library

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 

* Package name: golang-github-theupdateframework-go-tuf-v2
  Version : 2.0.2-1
  Upstream Author : The Update Framework (TUF)
* URL : https://github.com/theupdateframework/go-tuf
* License : Apache-2.0
  Programming Lang: Go
  Description : Go implementation of The Update Framework (TUF)

 The Update Framework (TUF) helps developers maintain the security of software
 update systems, providing protection even against attackers that compromise
 the repository or signing keys. TUF provides a flexible framework and
 specification that developers can adopt into any software update system.

I hope to maintain this package as part of Debian Go Packaging Team:

https://salsa.debian.org/go-team/packages/golang-github-theupdateframework-go-tuf-v2

The current Debian package golang-github-theupdateframework-go-tuf is
for the old v0.x API, quoting upstream:

 The legacy go-tuf (v0.7.0) (https://github.com/theupdateframework/go-
 tuf/tree/v0.7.0) codebase was difficult to maintain and prone to errors
 due to its initial design decisions. Now it is considered deprecated in
 favour of go-tuf v2 (originaly from rdimitrov/go-tuf-metadata
 (https://github.com/rdimitrov/go-tuf-metadata)) which started from the
 idea of providing a Go implementation of TUF that is heavily influenced
 by the design decisions made in python-tuf
 (https://github.com/theupdateframework/python-tuf).

Indeed, I tried rebuilding the reverse dependencies of this package with
v2.x and while most packages actually built, there are some that fails
due to TUF v0 vs v2:

https://salsa.debian.org/jas/golang-github-theupdateframework-go-tuf/-/pipelines/751423

Since the package has a different license and looks like a complete
rewrite to me, I think it makes sense to have two separate Debian
packages for it.

/Simon


signature.asc
Description: PGP signature

Bug#1085858: ITP: golang-github-digitorus-timestamp -- Time-Stamp Protocol (TSP) implementation for Go as specified in RFC3161

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 

* Package name: golang-github-digitorus-timestamp
  Version : 0.0~git20231217.220c5c2-1
  Upstream Author : Digitorus
* URL : https://github.com/digitorus/timestamp
* License : BSD-2-clause
  Programming Lang: Go
  Description : Time-Stamp Protocol (TSP) implementation for Go as 
specified in RFC3161

 RFC3161 Time-Stamp Protocol (TSP) package for Go
 .
 The package timestamp implements the Time-Stamp Protocol (TSP) as
 specified in RFC3161 (Internet X.509 Public Key Infrastructure Time-Stamp
 Protocol (TSP)).

I hope to maintain this package as part of Debian Go Packaging Team:

https://salsa.debian.org/go-team/packages/golang-github-digitorus-timestamp

/Simon


signature.asc
Description: PGP signature

Bug#1085857: ITP: golang-github-digitorus-pkcs7 -- Implements a subset of PKCS#7/Cryptographic Message Syntax (rfc2315, rfc5652)

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 

* Package name: golang-github-digitorus-pkcs7
  Version : 0.0~git20230818.3a137a8-1
  Upstream Author : Digitorus
* URL : https://github.com/digitorus/pkcs7
* License : Expat
  Programming Lang: Go
  Description : Implements a subset of PKCS#7/Cryptographic Message Syntax 
(rfc2315, rfc5652)

 pkcs7 implements parsing and creating signed and enveloped messages.
 .
 This is a fork of fullsailor/pkcs7 (https://github.com/fullsailor/pkcs7)

I hope to maintain this package as part of Debian Go Packaging Team:

https://salsa.debian.org/go-team/packages/golang-github-digitorus-pkcs7

I noticed that fullsailor/pkcs7 is already packaged in Debian, but
looking at the code, this package has evolved a lot.  I doubt that they
are still compatible in any reliable way.

/Simon


signature.asc
Description: PGP signature

Bug#1085856: ITP: golang-github-in-toto-attestation -- in-toto Attestation Framework

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 

* Package name: golang-github-in-toto-attestation
  Version : 1.1.0-1
  Upstream Author : in-toto
* URL : https://github.com/in-toto/attestation
* License : Apache-2.0
  Programming Lang: Go
  Description : in-toto Attestation Framework

 in-toto Attestation Framework
 .
 The in-toto Attestation Framework provides a specification for generating
 verifiable claims about any aspect of how a piece of software is
 produced. Consumers or users of software can then validate the origins
 of the software, and establish trust in its supply chain, using in-toto
 attestations.

I hope to maintain this package as part of Debian Go Packaging Team:

https://salsa.debian.org/go-team/packages/golang-github-in-toto-attestation

/Simon


signature.asc
Description: PGP signature

Bug#1085855: ITP: timestamp-authority -- RFC3161 Timestamp Authority

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 

* Package name: timestamp-authority
  Version : 1.2.3-1
  Upstream Author : sigstore
* URL : https://github.com/sigstore/timestamp-authority
* License : Apache-2.0
  Programming Lang: Go
  Description : RFC3161 Timestamp Authority

 Sigstore Timestamp Authority
 .
 A service for issuing RFC 3161 timestamps
 (https://datatracker.ietf.org/doc/html/rfc3161).
 .
 Timestamps conform to the RFC 3628 policy
 (https://datatracker.ietf.org/doc/html/rfc3628). The timestamp structure
 conforms to the updates in RFC 5816
 (https://datatracker.ietf.org/doc/rfc5816).
 .
 Security model
 .
 Trusted timestamping
 (https://en.wikipedia.org/wiki/Trusted_timestamping) is a process that
 has been around for some time. It provides a timestamp record of when a
 document was created or modified.
 .
 A timestamp authority creates signed timestamps using public key
 infrastructure. The operator of the timestamp authority must secure the
 signing key material to prevent unauthorized timestamp signing.
 .
 A timestamp authority should also verify its own clock. We provide a
 configuration to periodically check the current time against well-known
 NTP sources.
 .
 Timestamping within Sigstore
 .
 Timestamps are a critical component of Rekor
 (https://github.com/sigstore/rekor), Sigstore's signature transparency
 log. Timestamps are used to verify short-lived certificates. Currently,
 the timestamp comes from Rekor's own internal clock, which is not
 externally verifiable or immutable. Using signed timestamps issued from
 timestamp authorities mitigates the risk of Rekor's clock being
 manipulated.
 .
 As a artifact signer, you can:
 .
  * Generate a signature over an artifact
  * Fetch a timestamp for that signature (more below in What to sign)
  * Upload the signature, artifact hash, and certificate to Rekor
(hashedrekord record type)
  * Upload the timestamp to Rekor (rfc3161 record type)
* This step is important because it makes the timestamps publicly
auditable

I hope to maintain this package as part of Debian Go Packaging Team:

https://salsa.debian.org/go-team/packages/timestamp-authority

/Simon


signature.asc
Description: PGP signature

Bug#1085854: ITP: sigstore-go -- Go library for Sigstore signing and verification

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 

* Package name: sigstore-go
  Version : 0.6.2-1
  Upstream Author : sigstore
* URL : https://github.com/sigstore/sigstore-go
* License : Apache-2.0
  Programming Lang: Go
  Description : Go library for Sigstore signing and verification

 sigstore-go
 .
 A client library for Sigstore (https://www.sigstore.dev/), written in
 Go.
 .
 Features:
 .
  * Signing and verification of Sigstore bundles
(https://github.com/sigstore/protobuf-
specs/blob/main/protos/sigstore_bundle.proto) compliant with Sigstore
Client Spec
  * Verification of raw Sigstore signatures by creating bundles for them
(see conformance tests (/cmd/conformance/main.go) for example)
  * Signing and verifying with a Timestamp Authority (TSA)
  * Signing and verifying (offline or online) with Rekor (Artifact
Transparency Log)
  * Structured verification results including certificate metadata
  * TUF support
  * Verification support for custom trusted root
(https://github.com/sigstore/protobuf-
specs/blob/main/protos/sigstore_trustroot.proto)
  * Basic CLI and examples
 .
 There is not built-in support for signing with a KMS or other bring-your-
 own-key; however you can easily add support by implementing your own
 version of the interface pkg/sign/keys.go:Keypair.
 .
 For an example of how to use this library, see the verification
 documentation (/docs/verification.md), the CLI cmd/sigstore-go
 (/cmd/sigstore-go/main.go), or the CLI examples below. Note that the CLI
 is to demonstrate how to use the library, and not intended as a fully-
 featured Sigstore CLI like cosign (https://github.com/sigstore/cosign).
 .
 Background
 .
 Sigstore already has a canonical Go client implementation, cosign
 (https://github.com/sigstore/cosign), which was developed with a focus
 on container image signing/verification. It has a rich CLI and a long
 legacy of features and development. sigstore-go is a more minimal and
 friendly API for integrating Go code with Sigstore, with a focus on the
 newly specified data structures in sigstore/protobuf-specs
 (https://github.com/sigstore/protobuf-specs). sigstore-go attempts to
 minimize the dependency tree for simple signing and verification tasks,
 omitting KMS support and container image verification, and we intend to
 refactor parts of cosign to depend on sigstore-go.

I hope to maintain this package as part of Debian Go Packaging Team:

https://salsa.debian.org/go-team/packages/sigstore-go

/Simon


signature.asc
Description: PGP signature

Re: Will i386 released for Trixie and if no can we stop working on it now?

[Simon Josefsson]

Sebastiaan Couwenberg  writes:

> On 10/14/24 2:42 PM, Charles Plessy wrote:
>> The package names starts with r-cran.
>
> Time is better spent on updating the tooling to add
> architecture-is-64-bit to the build dependencies for those packages,
> as well as generating RM bug reports for the partial removals on [i386
> armel armhf].

Is that the recommended solution for packages that wants to stop
supporting 32-bit architectures?  Some of the *-wrapper packages (e.g.,
socket-wrapper, see #1069450) became FTBFS after the t64 upload, and I
haven't seen any solution except to stop supporting those architectures.
I used the 'not-supported-on [armel armhf]' approach instead of
'architecture-is-64-bit' in my last upload, but there are still reverse
dependencies on armel/armhf.  Could there be some automatic magic to
transitively remove 32-bit binaries of packages where the most recent
upload of a package uses 'architecture-is-64-bit'?  The manual work
involved to remove packages from the archive leads to people disabling
self tests on those architectures to avoid build failures, but this just
masks the problem that the package no longer works reliably on
armel/armhf.  The number of reverse dependencies for me is manageable,
but the consequences of the i386 decision for Charles seems sub-optimal.

/Simon


signature.asc
Description: PGP signature

Re: signify and signify-openbsd names

[Simon Josefsson]

Thanks for review!  I tried to revise the plan below, does this work?

I think we should compare this plan to simply remove the 'signify'
package, but haven't fleshed out that plan yet.

/Simon

x) Take current non-OpenBSD 'signify' source package and upload NEW
'signify-mail' package, say version 1.14-8 (?), that provides
/usr/bin/signify-mail instead of /usr/bin/signify, and has d/control:

Source: signify-mail
...
Package: signify-mail
Replaces: signify (<= 1.14-7)
Conflicts: signify (<= 1.14-7)
...
Package: signify
Depends: signify-mail
Breaks: signify (<= 1.14-7)
Replaces: signify (<= 1.14-7)
Provides: signify

x) Normal archive cleanup should remove the old 'signify' package.  If
this doesn't happen, once 'signify-mail' has entered testing, open a
ftp.debian.org RM bug for the old package 'signify' to get the binary
packages removed from unstable.

x) File a wishlist bug for 'signify-openbsd' (with patch) to ALSO
provide /usr/bin/signify (hardlink?), and to add a:

Conflicts: signify (<= 1.14-7)

Could it be a 'Conflicts: signify' to get the transitional dummy package
removed after installing 'signify-openbsd'?  Or does that just break
upgrades?

x) File a bug to suggest a trixie release note saying that the
non-OpenBSD 'signify' package has been rename to 'signify-mail', and
provides /usr/bin/signify-mail instead.  Say that the 'signify-openbsd'
package now provide /usr/bin/signify.  Also say that for trixie+1 the
intention is for the OpenBSD signify package to ship a 'signify' package
that provide /usr/bin/signify.

x) Open a bug for 'signify-mail' to say that the transitional package
should be removed in trixie+1.

x) Open a wishlist bug for 'signify-openbsd' (with patch) to track that
in trixie+1 (+2?) it should provide a 'Package: signify' that has
/usr/bin/signify and to add:

Conflicts: signify (<= 1.14-7)
Replaces: signify (<= 1.14-7)

The 'signify-openbsd' binary package should be left around as a empty
dummy package for transitions to the new 'signify' binary package:

Package: signify-openbsd
Depends: signify
Breaks: signify-openbsd (<= x.y.z)
Replaces: signify (<= x.y.z)
Provides: signify

Not sure when it makes sense to drop /usr/bin/signify-openbsd from the
'signify' package?  trixie+1?

x) OPTIONAL: Open a wishlist bug for 'signify-openbsd' to, after trixie,
upload itself as a NEW 'signify' source package and to ask for removal
of the old 'signify-openbsd' source package.  It was suggested this can
trigger BTS bugs, so may not be worth doing.  It doesn't really gain
anything except aesthetics.

x) Open a wishlist bug for the OpenBSD signify package to remove the
transitional 'signify-openbsd' package for trixie+2 (+3?).  The
/usr/bin/signify-openbsd name is then also removed.


signature.asc
Description: PGP signature

Re: signify and signify-openbsd names

[Simon Josefsson]

Paul Gevers  writes:

> Hi
>
> On 08-10-2024 09:01, Simon Josefsson wrote:
>> 3) Open a wishlist bug for 'signify-openbsd' with a patch to provide a
>> 'Package: signify' that has /usr/bin/signify and to add:
>
> Do I understand correctly that signify-mail will also provide a
> /usr/bin/signify?

Yes that was my idea.  But maybe that is a bad idea.

> That's not allowed if the binaries have different functionality [1],
> not even if the packages conflict. So apart from the name of the
> packages, also the path needs to adapted.

Good catch.  This seems to warrant a trixie release note, to say that
the non-OpenBSD 'signify' package's '/usr/bin/signify' has been renamed
to '/usr/bin/signify-mail', and (hopefully also) that the plan for
trixie+1 is for the OpenBSD 'signify-openbsd' source package to provide
a /usr/bin/signify.  That would also make the two packages
co-installable going forward, which is good.  This will break people's
scripts, unless they read the release notes and modify their calls to
'signify' into 'signify-mail'.

Interestingly that our processes makes something this simple (renaming a
tool) so complicated to achieve.

Another option is to remove the current non-OpenBSD 'signify' package
that hasn't seen a single update in upstream CVS since 2004.  What is
the criteria for keeping a package around when doing so requires a lot
of work from people who have no interest in the particular package, and
nobody who is interested in the particular package steps up to do the
work?  Still, the goal of having a OpenBSD /usr/bin/signify is worth
spending time on this for me.

I'll try to provide a revised plan after parsing Simon Richter's e-mail.
Or maybe two plans, one for removal, and one for renaming the package
and tool.

/Simon


signature.asc
Description: PGP signature

Re: signify and signify-openbsd names

[Simon Josefsson]

Attempting to summarize this thread into actions - please correct me
when I misunderstood:

1) Take current non-OpenBSD 'signify' source package and upload NEW
'signify-mail' with d/control modified as:

Source: signify-mail
...
Package: signify-mail
Replaces: signify (<= 1.14-7)

Do we need 'Breaks: signify (<= 1.14-7)' too?  Conflicts?

I've re-read chapter 7 of the policy manual again, but I have read it so
many times before and still don't feel confident about what it actually
means.  https://www.debian.org/doc/debian-policy/ch-relationships.html

2) Once 'signify-mail' has entered testing, open a ftp.debian.org RM bug
for 'signify' to get the binary packages removed from testing.

3) Open a wishlist bug for 'signify-openbsd' with a patch to provide a
'Package: signify' that has /usr/bin/signify and to add:

Conflicts: signify (<= 1.14-7), signify-mail

Is a similar Breaks needed too?

The 'signify-openbsd' binary package should be left around as a empty
dummy package for transitions to the new 'signify' binary package.

4) Uploading source package 'signify-openbsd' to NEW as 'signify', and
then ask for removal of the old 'signify-openbsd' source package.  This
is nice but optional.  It was suggested this can trigger BTS bugs.  It
may be best to wait until at least trixie+1.  This doesn't affect users
so is more of an developer aesthetic concern, which may suggest it isn't
worth doing at all.

/Simon


signature.asc
Description: PGP signature

Re: Alternative signature mechanisms for upstream source verification

[Simon Josefsson]

Otto Kekäläinen  writes:

>> No objections to have this kind of capability, but I still strongly
>> believe that importing tar archives is highly suboptimal and directly
>> branching off the upstream git repository is an highly superior workflow
>> and should be used as much as possible.
>>
>> This being said, I maintain some packages which are released by the
>> upstream maintainers only as tar archives (because OpenBSD).
>
> Also some projects release tarballs with extra additions that are not
> in the same git, or they strip away directories/files that are in git,
> but are irrelevant for users. If upstreams do that, then their intent
> is that downstreams are better off consuming the tarballs.

Indeed.  Some projects also release both variants -- for example
'libntlm' and 'oath-toolkit' upstream releases both traditional autoconf
tarballs and minimal source-only 'git archive' style tarballs.

> This is not a problem though, We can have the best of both, as
> git-buildpackage supports dual import from both upstream git and
> tarball to maximize supply chain auditability.
>
> You can see this in action in e.g.
> https://salsa.debian.org/mariadb-team/mariadb-server/-/network/debian%2Flatest?extended_sha1=f134a53ffcaad16eabedb30809837d5ee8170bc8&filter_ref=1
> The upstream branch 11.4 and tag mariadb-11.4.3 has the upstream git
> release contents, while the branch upstream/latest and tag
> upstream/11.4.3 shows the contents of the release tarball. The diff
> between these two branches shows how the upstream tarball differs from
> the upstream git commit at the time. The git side can be verified with
> git tag signature, and the tarball side is verified by tarball
> signature (thanks to also pristine-tar being used). This
> upstream/latest was then merged on debian/latest, which has git tags
> signed by Debian maintainer.
...
> If you want to see the details, see gbp.conf in the package
> (https://salsa.debian.org/mariadb-team/mariadb-server/-/blob/debian/latest/debian/gbp.conf).

That setup sounds nice!  What is your workflow to import a new upstream
release?

I see the watch file still points to the tarball:

https://salsa.debian.org/mariadb-team/mariadb-server/-/blob/debian/latest/debian/watch

Would it be possible to extend the debian/watch syntax to be able to
express both a tarball and upstream git branch?

Then 'gbp import-orig' would import both the tarball and sync git, after
performing PGP tarball verification and Git branch signature
verification.  I suppose you are doing that manually somehow now?

/Simon


signature.asc
Description: PGP signature

Re: signify and signify-openbsd names

[Simon Josefsson]

Ben Hutchings  writes:

> On Sat, 2024-10-05 at 20:15 +0200, Simon Josefsson wrote:
>> Ben Hutchings  writes:
>> 
>> > On Sat, 2024-10-05 at 12:31 +0200, Simon Josefsson wrote:
>> > [...]
>> > > This will rename the binary package to 'signify-mail', as suggested in
>> > > the first bug report above, and add a 'signify (<< 1.14-8~)' Replaces
>> > > header.
>> > > 
>> > > Is anything more required here?
>> > [...]
>> > 
>> > Yes, I think you should also rename the source package signify.
>> 
>> I think that would be nice from a human namespace perspective but I
>> don't know if Debian have any documented process for doing that.
>
> I am not aware of one.
>
>> Can
>> anyone find a pointer to relevant documentation?  What is the process?
>> Upload 'signify' to NEW again as 'signify-mail', and then ask for
>> removal of the 'signify'?  Can the source package name then be re-used
>> by 'signify-openbsd'?
>
> I believe that should work.  You would also ask for removal of the
> 'signify-openbsd' source package at the end of the process.
>
>> Or is there a rename operation policy, asking for
>> 'signify' to be renamed to 'signify-mail', and 'signify-openbsd' renamed
>> to 'signify'?
>
> I'm fairly sure there's no support for this in Debian infrastructure
> (dak or debbugs).
>
>> Doing renames is confusing for a long-term perspective,
>> how is that piece of meta-information recorded and where?  Is there any
>> earlier examples of a source package rename?
> [...]
>
> It's recorded in the changelog and, so far as I know, nowhere else.
>
> In 2012 the linux-2.6 source package was renamed to linux.  It had to
> go through NEW, but that was needed for every ABI bump anyway.  We had
> to track bugs against both source package names for several years
> (until EOL of the last release with linux-2.6).

I agree in principle, but I wonder if going through the effort of
introducing a new source package 'signify-mail' and removing the current
'signify' will give us anything beyond doing the QA package upload to
rename the binary package.

The only advantage I can identify seems to be if the 'signify-openbsd'
source package would then be able to be renamed to 'signify'.  But is
that possible?  Are there any earlier examples of re-use of the same
source package name, but for a different package?  The linux-2.6 vs
linux analogy is not identical, it is the same source package and there
were no source package namespace re-use happening.

This all assumes Tomasz as maintainer is interested in renaming his
'signify-openbsd' to 'signify'.

/Simon


signature.asc
Description: PGP signature

Re: signify and signify-openbsd names

[Simon Josefsson]

Ben Hutchings  writes:

> On Sat, 2024-10-05 at 12:31 +0200, Simon Josefsson wrote:
> [...]
>> This will rename the binary package to 'signify-mail', as suggested in
>> the first bug report above, and add a 'signify (<< 1.14-8~)' Replaces
>> header.
>> 
>> Is anything more required here?
> [...]
>
> Yes, I think you should also rename the source package signify.

I think that would be nice from a human namespace perspective but I
don't know if Debian have any documented process for doing that.  Can
anyone find a pointer to relevant documentation?  What is the process?
Upload 'signify' to NEW again as 'signify-mail', and then ask for
removal of the 'signify'?  Can the source package name then be re-used
by 'signify-openbsd'?  Or is there a rename operation policy, asking for
'signify' to be renamed to 'signify-mail', and 'signify-openbsd' renamed
to 'signify'?  Doing renames is confusing for a long-term perspective,
how is that piece of meta-information recorded and where?  Is there any
earlier examples of a source package rename?

> Debbugs doesn't always properly distinguish source and binary package
> names.  This goes badly when there are a source and binary package of
> the same name, but the binary package is built by a different source
> package.
>
> And of course, a situation like that is also confusing to humans.

+1

/Simon


signature.asc
Description: PGP signature

Re: signify and signify-openbsd names

[Simon Josefsson]

Hi

I would like that 'apt install signify' install OpenBSD's signify (from
the Debian 'signify-openbsd' package) and not the 2003 mail-related
signify perl script from the Debian 'signify' source package.

I would also like that /usr/bin/signify is OpenBSD's signify, after
doing the 'apt install signify', instead of /usr/bin/signify-openbsd.

As background please read:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=951010
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=738884

What do you think about the attached debdiff for the existing
(non-OpenBSD) 'signify' source package?

This will rename the binary package to 'signify-mail', as suggested in
the first bug report above, and add a 'signify (<< 1.14-8~)' Replaces
header.

Is anything more required here?  Any objections to an upload?  The
package seems to be orphan'ed and QA-maintained.  One alternative is to
remove the package, but that seems to involve more effort than a QA
upload.

The next step would be for the 'signify-openbsd' source package to
rename the 'signify-openbsd' binary package to 'signify'.  I believe it
should declare a Conflicts and Breaks towards 'signify (<< 1.14-8~)'.

The final step would be for signify-openbsd's 'signify' package to
rename the binary into /usr/bin/signify, and that would require adding a
Conflicts/Breaks towards all versions of signify-mail.

Tomasz, what do you think about this?

Adding debian-devel to get wider review of the
Replaces/Conflicts/version magic, this always tend to confuse me.

See pipeline for the 'signify' source package that I propose to upload:

https://salsa.debian.org/jas/signify/-/pipelines/742414

To make this happen for trixie, I don't see how to do it.  Anyone having
the old 'signify' package on their system would get OpenBSD's signify
instead of the new 'signify-mail' package after an upgrade.  Is that
problem really worth caring about?  I think a release note about this
could solve it.  I believe OpenBSD's 'signify' is important enough to
warrant this to be part of trixie instead of having to wait for t+1.

/Simon
diff -Nru signify-1.14/debian/changelog signify-1.14/debian/changelog
--- signify-1.14/debian/changelog	2020-06-05 21:06:41.0 +0200
+++ signify-1.14/debian/changelog	2024-10-05 11:58:55.0 +0200
@@ -1,3 +1,14 @@
+signify (1.14-8) unstable; urgency=medium
+
+  * QA upload.
+  * Rename binary package signify to signify-mail, adding Replaces
+field.
+  * Standards-Version: 4.7.0.
+  * Fix lintian update-debian-copyright.
+  * d/t/control: Use new package name.
+
+ -- Simon Josefsson   Sat, 05 Oct 2024 11:58:55 +0200
+
 signify (1.14-7) unstable; urgency=medium
 
   * QA upload.
diff -Nru signify-1.14/debian/control signify-1.14/debian/control
--- signify-1.14/debian/control	2020-06-05 21:06:41.0 +0200
+++ signify-1.14/debian/control	2024-10-05 11:58:55.0 +0200
@@ -4,14 +4,15 @@
 Build-Depends: debhelper-compat (= 13)
 Maintainer: Debian QA Group 
 Rules-Requires-Root: no
-Standards-Version: 4.5.0
+Standards-Version: 4.7.0
 Testsuite: autopkgtest-pkg-perl
 Vcs-Browser: https://salsa.debian.org/debian/signify
 Vcs-Git: https://salsa.debian.org/debian/signify.git
 Homepage: http://signify.sf.net
 
-Package: signify
+Package: signify-mail
 Depends: ${misc:Depends}, ${perl:Depends}
+Replaces: signify (<< 1.14-8~)
 Architecture: all
 Description: Automatic, semi-random ".signature" rotator/generator
  Signify is a neat little program that allows a random signature to be
diff -Nru signify-1.14/debian/copyright signify-1.14/debian/copyright
--- signify-1.14/debian/copyright	2020-06-05 21:06:41.0 +0200
+++ signify-1.14/debian/copyright	2024-10-05 11:58:55.0 +0200
@@ -10,6 +10,7 @@
 Files: debian/*
 Copyright: 1996-2004 Brian White 
2015 Mattia Rizzolo 
+	   2024 Simon Josefsson 
 License: public-domain
 
 License: public-domain
diff -Nru signify-1.14/debian/rules signify-1.14/debian/rules
--- signify-1.14/debian/rules	2020-06-05 21:06:41.0 +0200
+++ signify-1.14/debian/rules	2024-10-05 11:58:55.0 +0200
@@ -1,6 +1,6 @@
 #! /usr/bin/make -f
 
-p := signify
+p := signify-mail
 
 %:
 	dh $@
diff -Nru signify-1.14/debian/tests/control signify-1.14/debian/tests/control
--- signify-1.14/debian/tests/control	2020-06-05 21:06:41.0 +0200
+++ signify-1.14/debian/tests/control	2024-10-05 11:58:55.0 +0200
@@ -1 +1,2 @@
 Test-Command: signify --input=debian/tests/files/filetest.txt
+Depends: signify-mail


signature.asc
Description: PGP signature

Re: Alternative signature mechanisms for upstream source verification

[Simon Josefsson]

Stefano Rivera  writes:

> Should we expand this to include some of these new mechanisms?
> Things brought up in the debian-python thread include:
> 1. sigstore https://docs.sigstore.dev/
> 2. ssh signatures
> 3. signify https://man.openbsd.org/signify.1

+1

I believe all signatures we trust should be encoded in a non-mutable
transparency log like Sigstore/Sigsum etc.  But the first step towards
that is to add support for verifying that property.

> There is a general trend towards getting upstream sources from Git
> rather than tarballs in Debian, but we're a long way from moving across
> completely, or even finding consensus to do so.
> These signature mechanisms can generally be applied to git commits as
> well as tarballs.

Signatures of git commits is the same as a signature on a SHA1 object
which is broken for authentication purposes.  But it is possible to
discuss these issues separately, paving the way for git commit signing
to be trustworthy when GitHub/GitLab moves to SHA256.

/Simon


signature.asc
Description: PGP signature

Bug#1082817: ITP: golang-github-foxboron-go-tpm-keyfiles -- TPM 2.0 TSS keyfile library

[Simon Josefsson]

Package: wnpp
Severity: wishlist

* Package name: golang-github-foxboron-go-tpm-keyfiles
  Version : 0.0~git20240805.f870d6f-1
  Upstream Author : Morten Linderud
* URL : https://github.com/foxboron/go-tpm-keyfiles
* License : Expat
  Programming Lang: Go
  Description : TPM 2.0 TSS keyfile library

 go-tpm-keyfile
 .
 Implements the ASN.1 Specification for TPM 2.0 Key Files.
 .
 (https://www.hansenpartnership.com/draft-bottomley-tpm2-keys.html)

/Simon


signature.asc
Description: PGP signature

Bug#1082814: ITP: ssh-tpm-agent -- ssh-agent for TPMs

[Simon Josefsson]

Package: wnpp
Severity: wishlist

* Package name: ssh-tpm-agent
  Version : 0.6.0-1
  Upstream Author : Morten Linderud
* URL : https://github.com/foxboron/ssh-tpm-agent
* License : Expat
  Programming Lang: Go
  Description : ssh-agent for TPMs

 SSH agent for TPM
 .
 ssh-tpm-agent is a ssh-agent compatible agent that allows keys to be
 created by the Trusted Platform Module (TPM) for authentication towards
 ssh servers.
 .
 TPM sealed keys are private keys created inside the Trusted Platform
 Module (TPM) and sealed in .tpm suffixed files. They are bound to the
 hardware they are produced on and can't be transferred to other
 machines.
 .
 This allows you to utilize a native client instead of having to side
 load existing PKCS11 libraries into the ssh-agent and/or ssh client.
 .
 The project uses TPM 2.0 Key Files
 (https://www.hansenpartnership.com/draft-bottomley-tpm2-keys.html)
 implemented through the go-tpm-keyfiles (https://github.com/Foxboron/go-tpm-
 keyfiles) project.
 .
 Features
 .
  * A working ssh-agent.
  * Create shielded ssh keys on the TPM.
  * Creation of remotely wrapped SSH keys for import.
  * PIN support, dictionary attack protection from the TPM allows you to
use low entropy PINs instead of passphrases.
  * TPM session encryption.
  * Proxy support towards other ssh-agent servers for fallbacks.

/Simon


signature.asc
Description: PGP signature

Bug#1079669: ITP: libntruprime -- Streamlined NTRU Prime (sntrup) microlibrary

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 
X-Debbugs-Cc: debian-devel@lists.debian.org

* Package name: libntruprime
  Version : 20240825
  Upstream Authort: Daniel J. Bernstein
* URL : https://libntruprime.cr.yp.to/
* License : LicenseRef-PD-hp OR CC0-1.0 OR 0BSD OR MIT-0 OR MIT
  Programming Lang: C
  Description : Streamlined NTRU Prime (sntrup) microlibrary

libntruprime is a microlibrary for the Streamlined NTRU Prime
cryptosystem. Streamlined NTRU Prime (sntrup) is a lattice-based
cryptosystem with the following features:

 - Stability: Almost all details of sntrup match a May 2016
   publication. The only exceptions are small changes to encoding and
   hashing published in April 2019.

 - Patent-freeness: April 2019 predates almost all post-quantum
   patents. Analyses of various lattice patents filed before April 2019
   indicate no problems for sntrup.

 - Deployment: The popular OpenSSH tool switched to sntrup761 by default
   in April 2022, following initial integration of sntrup into TinySSH.

 - Affordability: Keys and ciphertexts are about 1KB for sntrup761, and
   computations are fast.

 - Careful design: Subject to the requirement of being a small
   lattice-based cryptosystem, sntrup is systematically designed to
   eliminate unnecessary complications in security review. It eliminates
   decryption failures, for example, and eliminates cyclotomics. The
   cryptosystem has never needed a security patch.

 - Risk management: A much higher sntrup1277 security level is fully
   supported, and is recommended whenever 2KB keys and ciphertexts are
   affordable, to reduce risks from improvements in lattice attacks.

 - Flexibility: The sntrup design allows a full spectrum of tradeoffs
   between size and security level, so applications with intermediate
   size limits aren't forced into much lower security levels. Six
   different sizes have been selected for support.

libntruprime has a very simple stateless API based on the SUPERCOP API,
with wire-format inputs and outputs, providing functions that directly
match the KEM operations provided by the sntrup specification, such as
functions

sntrup1277_keypair
sntrup1277_enc
sntrup1277_dec

for the sntrup1277 KEM.

Internally, libntruprime includes implementations designed to work
portably across CPUs, and implementations designed for higher
performance on Intel/AMD CPUs with AVX2 instructions. libntruprime
includes automatic run-time selection of implementations.

libntruprime is intended to be called by larger multi-function libraries
(such as traditional cryptographic libraries), including libraries in
other languages via FFI. The idea is that libntruprime takes
responsibility for the details of sntrup computation, including
optimization, timing-attack protection, and (in ongoing work)
verification, freeing up the calling libraries to concentrate on
application-specific needs such as protocol integration. Applications
can also call libntruprime directly.

I hope to maintain this at https://salsa.debian.org/jas/libntruprime

/Simon


signature.asc
Description: PGP signature

Re: Accepting DEP14?

[Simon Josefsson]

Andrey Rakhmatullin  writes:

> On Sat, Aug 17, 2024 at 12:20:16PM +0200, Andreas Tille wrote:
>> My personal preference would be if we make a pristine-tar branch default
>> since this is what I observed in the wide majority of cases.
>
> Note that there are different opionons whether pristine-tar is
> needed/viable/useful. There is at least one objective fact that it's hard
> to keep working.

Did anyone consider a way to only store meta-information about the
source tarballs in the debian/ tree?

I'm thinking the SHA-256 checksum of the tarball should be recorded and
be part of the signed debian.tar.xz upload.

For example, libidn2's debian/source/artifact could contain:

Checksums-SHA256: 
4c21a791b610b9519b9d0e12b8097bf2f359b12f8dd92647611a929e6bfd7d64 2155214 
libidn2_2.3.7.orig.tar.gz

Or something like that.

If the debian/ tree contained a SHA-256 checksum of the upstream release
artifact, it could be used to locate and retrieve the tarballs using any
mechanism available, reducing the need for pristine-tar which is fragile
(how many checks that the checksum of the pristine-tar tarball matches
what's in the debian archive?).

It would also strengthen the integrity of the resulting archive, since
then there is some way to look at a *.debian.tar.* and git debian/
sub-directory and understand what the intended source code it applies to
are.

Manually curating the release artifact checksums is what many other
packaging systems do, and I think it is a good pattern.

/Simon


signature.asc
Description: PGP signature

Re: Accepting DEP14?

[Simon Josefsson]

Chris Hofstaedtler  writes:

> On Fri, 16 Aug 2024 23:36:31 +0200, gregor herrmann wrote:
>> IMO, and from discussions in the Debian Perl Group, the blocker is
>> the conversion of existing repos, both on salsa (which should be
>> doable via the API as suggested in the sketches mentioned above) and
>> also locally for hundreds of developer machines [git fails horribly
>> on the upstream/ → upstream/latest change].
>
> I want to echo this pain. When changing the layout it seems almost
> better to start from scratch.
>
> Additionally, in my opinion debian/latest is a mistake we should not
> recommend.

+1

I'm happily working on packages with many different git styles, and I'm
building packages for different suites and vendors: the 'debian/latest'
naming style has not contributed to my work flow.

I do understand the original rationale, I think, since it feels ugly to
put an experimental upload on a debian/unstable git branch.

However I think there are two kind of experimental uploads that are
different in nature:

   1) uploads to experimental that test things intended for sid quite
   soon (maybe architecture build testing like we just did for
   'libmceliece'), and

   2) complete odd-ball uploads that is known to break things, but
   needed for some other experimentation that may be multi-months while
   there could be uploads going into unstable (like we did for the Go
   grpc package during the last months).

I believe for the use-case 1) it is better to use debian/unstable
immediately and for 2) it is better to have one branch debian/unstable
and one branch debian/experimental.  I'm missing what positive effects
arise from using a debian/latest branch.

Maybe this could be clarified in DEP14 that 'debian/latest' is only
recommended for use instead of 'debian/unstable' for 1)-style uploads.
I still wonder if it is actually wortwhile to rescue 'debian/latest'
though.  I've often used 'debian/unstable' instead for this purpose,
since the 1)-style uploads are not uncommon for me.

I think the core problem is that there really is no reasonable concept
of "latest" branch for a Debian package.  There are just many different
versions targetted at different suites or vendors, each of them having
their own meaning of what is latest.

/Simon


signature.asc
Description: PGP signature

Re: Bug#1074175: netkit-rwho: remove for trixie?

[Simon Josefsson]

Hi.

Most tools from netkit are candidates for migration to GNU InetUtils,
and rwho(d) may be another one -- see email and bug report below.
Cc'ing debian-devel to have broader discussion.

First, I think we need to understand the rationale for doing anything
about 'netkit-rwho': do we want to do something because 1) it is not
maintained upstream? or 2) because it is an insecure design?, or 3)
something else?

I believe that like telnet and ftp the second argument is not convincing
enough: sometimes you need these implementations for various strange
things, and it is poor style to dictate what people must do with their
software.  The position I've taken in GNU InetUtils is that it is better
for users to offer maintained tools and include a notice that they are
insecure, rather to offer un-maintained tools and refuse to work further
on them because they are insecure, putting users into a worse situation
than before.  Some people may disagree, and instead believe it is better
to actively kill old things rather than continue support them.  This is
a subjective decision, and if people are willing to do the work to keep
things alive, I think it is better to let them than to refuse this
contribution.

So, are our reason for doing anything about netkit-rwho really because
netkit upstream is not maintained?

If so, then one option is to add a rwho(d) implementation to GNU
InetUtils and let that replace the netkit implementation in Debian.
Historically, netkit tools have often had unclear or weird license
situation, so my preference is to import rwho(d) from some of the BSD
and to make that build for a wide variety of architectures and platforms
like we do with other tools in GNU InetUtils.  The BSD implementations
are usually not intended to be portable, and often have some minor flaw
that makes them troublesome to build on Debian -- we fix those issues in
GNU InetUtils.

That said, introducing yet another fork into the ecosystem shouldn't be
done lightly, so we should explore some way to pool resources (like I've
tried to establish with tnftp(d) maintainers when we have joint bugs).
I haven't analyzed what rwho(d) implementations are out there.  I see
NetBSD/FreeBSD has one still in -current, but OpenBSD removed it during
5.x.  Are people aware of any other implementations worth considering?

What do you think?

/Simon

Gürkan Myczko  writes:

> rwho(d) is a design from a different time, when networks were
> trusted, and so on. It seems to me, we should and could stop
> shipping it for trixie.
>
> I'm raising this bug now, to:
> 1) establish awareness
>
> I was long aware of this, as I was using rwhod/ruptime when networks
> were not split into thousand networks...
>
> 2) auto-rm it from trixie
>
> I'd rather have a migration path, not binary compatible, but
> functionality compatible
>
> 3) give people time to chime in / secure replacements to show up
>
> Please have a look at https://github.com/alexmyczko/rutpime (there's
> an ITP for it, and it has
> been in new queue several times:
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1013361
>
> After a while I intend to clone this bug to ftp.debian.org for
> removal from unstable.
>
> Please do not remove it if possible. I really wish to have a migration
> path for this, but well
> we're waiting for ftp team.
>
> Best,
> Alex
>
> Chris
>
>


signature.asc
Description: PGP signature

Bug#1075715: ITP: cppi/1.18 -- adjusts or checks indentation of C and C++ preprocessor directives

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 
X-Debbugs-Cc: debian-devel@lists.debian.org

* Package name: cppi
  Version : 1.18-1
  Upstream Author : Jim Meyering, FSF, et al
* URL : https://www.gnu.org/software/cppi/
* License : GPL-3+
  Programming Lang: C
  Description : adjusts or checks indentation of C and C++ preprocessor 
directives

 GNU cppi adjusts or checks the indentation of C and C++ preprocessor
 directives in a file.
 .
 Indent the C preprocessor directives in FILE to reflect their nesting
 and ensure that there is exactly one space character between each #if,
 #elif, #define directive and the following token, and write the result
 to standard output.  The number of spaces between the `#' and the following
 directive must correspond to the level of nesting of that directive.

I intend to maintain this package on Salsa at
https://salsa.debian.org/debian/cppi

/Simon


signature.asc
Description: PGP signature

gzip --rsyncable (was: Re: De-vendoring gnulib in Debian packages)

[Simon Josefsson]

"Theodore Ts'o"  writes:

> On Sun, May 12, 2024 at 04:27:06PM +0200, Simon Josefsson wrote:
>> Going into detail, you use 'gzip -9n' but I use git-archive defaults
>> which is the same as -n aka --no-name.  I agree adding -9 aka --best is
>> an improvement.  Gnulib's maint.mk also add --rsyncable, would you agree
>> that this is also an improvement?
>
> I'm not convinced --rsyncable is an improvement.  It makes the
> compressed object slightly larger, and in exchange, if the compressed
> object changes slightly, it's possible that when you rsync the changed
> file, it might be more efficient.  But in the case of PGP signed
> release tarballs, the file is constant; it's never going to change,
> and even if there are slight changes between say, e2fsprogs v1.47.0
> and e2fsprogs v1.47.1, in practice, this is not something --rsyncable
> can take advantage of, unless you manually copy
> e2fsprogs-v1.47.0.tar.gz to e2fsprogs-v1.47.1.tar.bz, and then rsync
> e2fsprogs-v1.471.tar.g and I don't think anyone is doing this,
> either automatically or manually.
>
> That being said, --rsyncable is mostly harmless, so I don't have
> strong feelings about changing it to add or remove in someone's
> release workflow.

Your example had me convinced, and I thought some more about why we
really should keep using it as it consumes a small percentage more CPU
and disk space.  I have realized that another common operation is
storing and transfering _several_ different releases of e2fsprogs.  I
would suspect that most releases of software is fairly similar to the
previous release when uncompressed.  With gzip --rsyncable, the tarballs
should then be mostly similar.  Without --rsyncable, they will largely
be different if I understand correctly.  This affects dedup-able storage
and transfer methods, and some anecdotical evidence suggests this
improvement is significant - going from 215GB to 176GB vs 13GB:

https://gitlab.archlinux.org/archlinux/infrastructure/-/merge_requests/429

Maybe someone could do some experiment to see if there is substance to
this argument, its not clear to me that the example is comparable.
Storing/transferring several releases for the same software could add
significant savings for larger set of archives.

As the downside seems fairly small, and the potential upside may be
significant, I will use and recommend --rsyncable for git-archive
release tarballs:

   git archive --format=tar --prefix=$PACKAGE-$VERSION/ HEAD | \
  env GZIP= gzip --no-name --best --rsyncable \
  > $PACKAGE-$VERSION-src.tar.gz

/Simon


signature.asc
Description: PGP signature

Re: De-vendoring gnulib in Debian packages

[Simon Josefsson]

"Theodore Ts'o"  writes:

>> 1) Use upstream's PGP signed git-archive tarball.
>
> Here's how I do it in e2fsprogs which (a) makes the git-archive
> tarball be bit-for-bit reproducible given a particular git commit ID,
> and (b) minimizes the size of the tarball when stored using
> pristine-tar:
>
> https://github.com/tytso/e2fsprogs/blob/master/util/gen-git-tarball

Wow, written five years ago and basically the same thing that I suggest
(although you store pre-generated ./configure scripts in git).

Going into detail, you use 'gzip -9n' but I use git-archive defaults
which is the same as -n aka --no-name.  I agree adding -9 aka --best is
an improvement.  Gnulib's maint.mk also add --rsyncable, would you agree
that this is also an improvement?  Thus what I'm arriving at is this:

git archive --prefix=inetutils-$(git describe)/ HEAD |
   gzip --no-name --best --rsyncable > -o inetutils-$(git describe)-src.tar.gz

>> To reach our goals in the beginning of this post, this upstream tarball
>> has to be filtered to remove all pre-generated artifacts and vendored
>> code.  Use some mechanism, like the debian/copyright Files-Excluded
>> mechanism to remove them.  If you used a git-archive upstream tarball,
>> chances are higher that you won't have to do a lot of work especially
>> for pre-generated scripts.
>
> Why does it *has* to be filtered?  For the purposes of building, if
> you really want to nuke all of the pre-generated files, you can just
> move them out of the way at the beginning of the debian/rules run, and
> then move them back as part of "debian/rules clean".  Then you can use
> autoreconf -fi to your heart's content in debian/rules (modulo
> possibly breaking things if you insist on nuking aclocal.m4 and
> regenerating it without taking proper care, as discussed above).
>
> This also allows the *.orig.tar.gz to be the same as the upstream
> signed PGP tarball, which you've said is the ideal, no?

Right, there is no requirement for orig.tar.gz to be filtered.  But then
the outcome depends on upstream, and I don't think we can convince all
upstreams about these concerns.  Most upstream prefer to ship
pre-generated and vendored files in their tarballs, and will continue to
do so.  Let's assume upstream doesn't ship minimized tarballs that are
free from vendored or pre-generated files.  That's the case for most
upstream tarballs in Debian today (including e2fsprogs, openssh,
coreutils).  Without filtering that tarball we won't fulfil the goals I
mentioned in the beginning of my post.  The downsides with not filtering
include (somewhat repeating myself):

- Opens up for bugs causing pre-generated files not being re-generated
  even when they are used to build the package.  I think this is fairly
  common in Debian packages.  Making sure all pre-generated files are
  re-generated during build -- or confirming that the file is not used
  at all -- is tedious and fragile work.  Work that has to be done for
  every release.  Are you certain that ./configure is re-generated?  If
  it is not present you would notice.

- Auditing the pre-generated and vendored files for malicious content
  takes more time than not having to audit those files.  Especially if
  those files are not stored in upstream git.

- Pre-generated and vendored files trigger licensing concerns and
  require tedious work that doesn't improve the binary package
  deliverable.  Consider files like texinfo.tex for example, wouldn't it
  be better to strip that out of tarballs and not have to add it to
  debian/copyright?  If some code in a package, let's say getopt.c, is
  not used during build of the package, the license of that file doesn't
  have to be mentioned in debian/copyright if I understand correctly:
  https://www.debian.org/doc/debian-policy/ch-archive.html#s-pkgcopyright
  If in a few releases later, that file starts to get used during
  compilation, the package maintainer will likely not notice.  If it was
  filtered, the maintainer would notice.

The best is when upstream ship a tarball consistent with what I dream
*.orig.tar.gz should be: free of vendored and pre-generated files.
Debian package maintainers can take action before this happens, to reach
nice properties within Debian.  Maybe some upstream will adapt.

>> There is one design of gnulib that is important to understand: gnulib is
>> a source-only library and is not versioned and has no release tarballs.
>> Its release artifact is the git repository containing all the commits.
>> Packages like coreutils, gzip, tar etc pin to one particular commit of
>> gnulib.
>
> Note that how we treat gnulib is a bit differently from how we treat
> other C shared libraries, where we claim that *all* libraries must be
> dynamically linked, and that include source code by reference is
> against Debian Policy, precisely because of the toil needed to update
> all of the binary packages should some security vulnerability gets
> discovered in the library which is either linked statically o

Re: De-vendoring gnulib in Debian packages

[Simon Josefsson]

Bruno Haible  writes:

> Simon Josefsson wrote:
>> Finally, while this is somewhat gnulib specific, I think the practice
>> goes beyond gnulib
>
> Yes, gnulib-tool for modules written in C is similar to
>
>   * 'npm install' for JavaScript source code packages [1],
>   * 'cargo fetch' for Rust source code packages [2],
>
> except that gnulib-tool is simpler: it fetches from a single source location
> only.
>
> How does Debian handle these kinds of source-code dependencies?

I don't know the details but I believe those commands are turned into
local requests for source code, either vendored or previously packaged
in Debian.  No network access during builds.  Same for Go packages,
which I have some experience with, although for Go packages they lose
the strict versioning so if Go package X declare a depedency on package
Y version Z then on Debian it may build against version Z+1 or Z+2 which
may in theory break and was not upstream's intended or supported
configuration.  We have a circular dependency situation for some core Go
libraries in Debian right now due to this.

I think fundamentally the shift that causes challenges for distributions
may be dealing with packages dependencies that are version >= X to
package dependencies that are version = X.  If there is a desire to
support that, some new patterns of the work flow is needed.  Some
package maintainers reject this approach and refuse to co-operate with
those upstreams, but I'm not sure if this is a long-term winning
strategy: it often just lead to useful projects not being available
through distributions, and users suffers as a result.

/Simon


signature.asc
Description: PGP signature

De-vendoring gnulib in Debian packages

[Simon Josefsson]

r 2), then create a minimal source-only copy of the git
   archive and sign it yourself.  Could be done something like this:

   git clone https://git.savannah.gnu.org/git/inetutils.git
   cd inetutils/
   git archive --prefix=inetutils-v2.5/ -o inetutils-2.5-src.tar.gz v2.5
   # additional filtering of tarball may go here
   gpg -b inetutils-2.5-src.tar.gz

   This is your new upstream tarball.  To build this particular one, use
   ./bootstrap --no-git --gnulib-srcdir=/usr/share/gnulib.

4) Use upstream's git-archive tarball and PGP sign it.

   Download it using the GitHub or GitLab download link on the git tag
   like the cool kids.  If you did this on a sunny day, the downloaded
   tarball should be identical to the git-archive tarball and you can
   sign it if you are comfortable with this.

5) Use upstream's git-archive tarball.

   For those who want to join the really cool kids club.

6) Use upstream's tarball without PGP signature.

   This is quite common today.  It happens when upstream doesn't publish
   PGP signatures or the Debian maintainer doesn't care about them.

Regardless of mechanism, you should end up with a tarball that we call
the "upstream tarball".  Which approach is chosen is subjective and up
to the Debian package maintainer.  people have different opinions.
While I can't hide my own preferences I think we have to acknowledge
that there is no single uniform answer here.

To reach our goals in the beginning of this post, this upstream tarball
has to be filtered to remove all pre-generated artifacts and vendored
code.  Use some mechanism, like the debian/copyright Files-Excluded
mechanism to remove them.  If you used a git-archive upstream tarball,
chances are higher that you won't have to do a lot of work especially
for pre-generated scripts.

This filtered tarball will be the *.orig.tar.gz used to build the Debian
package.

Ideally you would like for the *.orig.tar.gz tarball to be as close as
possible to upstream's git repository for the tag release, minus any
pre-generated scripts or vendored gnulib files that upstream put into
git.  For collaborative upstreams, you could try to convince them to not
put pre-generated scripts and vendored gnulib files into git.

Auditing the upstream tarball to the *.orig.tar.gz should be simple, use
sha256sum or diffoscope to compare content.  In some ideal world this
could be bit-by-bit identical.  I'm hoping this can be the new best
recommended approach going forward.  This is only possible when upstream
agree with these concerns, and make an effort to publish such minimized
source-only tarballs.  This may be a pipe dream, just like Debian's
current best recommended approach for upstream PGP signed tarballs are
sometimes ignored.

You will now be faced with the challenge of building this tarball.  Your
existing debian/rules makefile will not work any more since it assumed
the existance of the pre-generated scripts and vendored gnulib files.
So you have to add the required tools as Build-Depends: and update the
debian/rules to build everything from source code.

For libntlm the essential diff between version 1.7-1, that used upstream
tarball with pre-generated content and gnulib code, and latest version
1.8-3 that builds from a minimal source-only tarball is small:

--- a/debian/control
+++ b/debian/control
@@ -6,6 +6,8 @@ Uploaders:
  Simon Josefsson ,
 Build-Depends:
  debhelper-compat (= 13),
+ git,
+ gnulib (>= 20240412~dfb7117+stable202401.20240408~aa0aa87-3~),
 Standards-Version: 4.6.2
 Section: libs
 Homepage: https://www.nongnu.org/libntlm/
--- a/debian/rules
+++ b/debian/rules
@@ -1,6 +1,16 @@
 #! /usr/bin/make -f
 
+include /usr/share/gnulib/debian/gnulib-dpkg.mk
+
 export DEB_BUILD_MAINT_OPTIONS = hardening=+all
 
 %:
-   dh $@ --builddirectory=build -X.la
+   dh $@ --without autoreconf --builddirectory=build
+
+pull:
+   ./bootstrap --gnulib-srcdir=$(GNULIB_DEB_DEBIAN_GNULIB) --pull
+
+gen:
+   ./bootstrap --gnulib-srcdir=$(GNULIB_DEB_DEBIAN_GNULIB) --gen
+
+execute_before_dh_auto_configure: dh_gnulib_clone pull dh_gnulib_patch gen

As you can see the essential part is to add a Build-Depends on the
gnulib Debian package to get the necessary gnulib code for building.  We
also disable dh_autoreconf since its approach is no longer necessary
(and hides problems), everything is built from source coming from Debian
or upstream.

There is one design of gnulib that is important to understand: gnulib is
a source-only library and is not versioned and has no release tarballs.
Its release artifact is the git repository containing all the commits.
Packages like coreutils, gzip, tar etc pin to one particular commit of
gnulib.  There is little coordination among packages which gnulib git
commit to use, and historically they typically use the latest gnulib git
commit that was published when the release manager prepared a release.
Usually the pinning happens through a 

archive.debian.org mirrors

[Simon Josefsson]

Hi

According to the mirror list

https://www.debian.org/distrib/archive

it should be possible to reach archive.debian.org via rsync, however
this fails for me.  Is this intentional, or can this be fixed?

Further it seems mirrors are out of sync.  I noticed that several
mirrors lack buster.  According to timestamps on archive.debian.org
these directories were added on 2024-03-10 -- is it usual for mirror
delays to be this long?

Seeing mirror functionality being uncertain, I have started to track
archive.debian.org as a Git-LFS repository on gitlab.com.  Until rsync
access against archive.debian.org is operational, this is based on
mirror's content which may be stale. I hope to figure out which rsync
mirror has the latest version, and keep this updated going forward.

https://gitlab.com/debdistutils/archives/debian/archive.debian.org

/Simon


signature.asc
Description: PGP signature

Re: Binary conflict between Midnight Commander and MinIO Client

[Simon Josefsson]

Philip Hands  writes:

> Mathias Gibbens  writes:
>
>> On Sun, 2024-04-21 at 18:47 +0200, Simon Josefsson wrote:
> ...
>>> /usr/libexec/minio-client/bin/mc -> /usr/bin/mcli
>
> Might I suggest that the link goes the other way, so that the symlink
> lives in /usr/bin?  That way the existence of the lib directory is
> somewhat self-documenting.
>
> If you're looking for examples, it seems that fd-find and binutils-avr
> do something like this (although they seem to be linking into ../lib/
> presumably for historical reasons).

It seems this approach could use some documentation, if we want to have
some consistent implementation of this idea in Debian.  While I usually
dislike symlinks in /usr/bin/ I think there is some advantages.  I'm
also not sure where a good path component to store such binaries are,
libexec or lib?  Or share?

/Simon


signature.asc
Description: PGP signature

Re: Binary conflict between Midnight Commander and MinIO Client

[Simon Josefsson]

Marco d'Itri  writes:

> On Apr 21, Mathias Gibbens  wrote:
>
>>   While that might work for them, it obviously doesn't at a higher
>> packaging level. Per Policy Section 10.1, I'm bringing this to d-devel
>> for any comments or suggestions on my plan for packaging the MinIO
>> Client. Following what several other distributions have done[2], I'm
>> planning to name the source/binary packages "minio-client" and the
>> binary provided from that package will be `mcli`.

+1

> Go for it, I think that there is no good solution for this case.
> Everybody who cares then will manually create a mc -> mcli symlink.

Several Homebrew packages uses an approach that I regard as superior to
what the debian ecosystem provides for this problem: putting files in a
path that users can add to their $PATH to get upstreams' desired binary
name, when there is a conflict with a historically established name.  So
for this example, minio-client could create a symlink like this

/usr/libexec/minio-client/bin/mc -> /usr/bin/mcli

and users who really want the upstream behaviour can solve this by
modifying environment variables.

/Simon


signature.asc
Description: PGP signature

Re: gnulib

[Simon Josefsson]

Jonas Smedegaard  writes:

> Quoting Simon Josefsson (2024-04-18 09:34:26)
>> Jonas Smedegaard  writes:
>> 
>> > That said, you are welcome to try nudge me if some concrete task
>> > emerges where you image I might be of help.
>> 
>> Thanks -- I'm moving this out of 921954@bugs and cc'ing debian-devel to
>> allow others to help and to allow you from not having to feel a need to
>> reply at all :)
>
> Thanks for releaving me.
>
> ...but then you bring up licensing, which has my special interest :-D

I am terribly sorry :-)

>> One of the things that bothered me with the gnulib Debian package that
>> I've been too afraid to touch is the debian/copyright file.  It triggers
>> a lot of lintian errors:
>> 
>> https://udd.debian.org/lintian/?packages=gnulib
>> 
>> For reference here is current debian/copyright:
>> 
>> https://salsa.debian.org/debian/gnulib/-/blob/debian/sid/debian/copyright
>> 
>> I've seen debian/clscan/ and ran the tools there, but I don't yet feel
>> comfortable patching things, and it didn't produce clean results even
>> for the last version in testing before I started to work on this
>> package, so I'm not convinced this toolchain is the best approach going
>> forward.
>
> When I took over maintenance my first thought was also to get rid of the
> clscan script, but then I realized how enormous a work it would be to
> approach it differently and wrapped my head around the script and
> adjusted it.
>
> Does it sound like you are in a similar situation now as I was, or is
> there something in particular that makes you consider abandoning
> clscan?

Yes you are right.  There is nothing in particular that I've found,
except that I don't understand how it is supposed to work and I felt
uncertain if it was worth wrapping my head around or not.

>> One problem is that lintian doesn't like [REF01] in lines like this:
>> 
>> License: FSFAP [REF01]
>
> I agree with lintian about the above (but we disagree on other things -
> see bug#786450). I am confident that the above syntax is incorrect:
> copyright format 1.0 requires a single-word shortname.

That is good to establish, and I wasn't even certain of that.  Then it
is clear that it is the gnulib debian/copyright file that should change.
And the discussion can move to what it should change into.

> If you are simply not fluent in perl, then perhaps reach out to the
> Debian perl team for help? Or perhaps look in the git history the tweaks
> that I made - perhaps those are of inspiration to whatever issue you are
> running into now?

I will try to do that -- and will experiment with it to see if I get an
improved copyright file out of it, maybe using a Comment: approach
instead of the invalid [REF01] approach.

/Simon


signature.asc
Description: PGP signature

Re: Status of the t64 transition

[Simon Josefsson]

Sebastian Ramacher  writes:

> Hi,
>
> as the progress on the t64 transition is slowing down, I want to give an
> overview of some of the remaining blockers that we need to tackle to get
> it unstuck. I tried to identify some clusters of issues, but there might
> be other classes of issues.

Thanks for update.  I haven't seen any response to my analysis of why
'shishi' shouldn't have t64-migrating, and lacking any response I
eventually made an upload to revert the renaming.  I've raised an Ubuntu
bug too since they haven't synced the updated package.  The Debian bug
is now closed, so it wasn't included in your list, and maybe everything
that can be hoped for is already achieved, but I wanted to mention as it
ought to be on the t64 radar.

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1062896
https://bugs.launchpad.net/ubuntu/+source/shishi/+bug/2061743
https://tracker.debian.org/pkg/shishi

/Simon


signature.asc
Description: PGP signature

Re: gnulib

[Simon Josefsson]

Jonas Smedegaard  writes:

> That said, you are welcome to try nudge me if some concrete task
> emerges where you image I might be of help.

Thanks -- I'm moving this out of 921954@bugs and cc'ing debian-devel to
allow others to help and to allow you from not having to feel a need to
reply at all :)

One of the things that bothered me with the gnulib Debian package that
I've been too afraid to touch is the debian/copyright file.  It triggers
a lot of lintian errors:

https://udd.debian.org/lintian/?packages=gnulib

For reference here is current debian/copyright:

https://salsa.debian.org/debian/gnulib/-/blob/debian/sid/debian/copyright

I've seen debian/clscan/ and ran the tools there, but I don't yet feel
comfortable patching things, and it didn't produce clean results even
for the last version in testing before I started to work on this
package, so I'm not convinced this toolchain is the best approach going
forward.

One problem is that lintian doesn't like [REF01] in lines like this:

License: FSFAP [REF01]

Is the reason why this is done that you want to record a full copy of
the actual text from the particular file AND some more information?
Sometimes there is a file X with the FSFAP license and some additional
text not part of the core FSFAP license, and another file Y that also
uses FSFAP but has some OTHER additional text that you want to record?

In some other packages, I've used the Comment: field like this for
situations like that.  Maybe it is applicable here?

Files: *
Copyright: 2016 Google LLC. All Rights Reserved.
   2022 Trillian Authors. All Rights Reserved.
   2016 The Kubernetes Authors.
   2017 Google LLC. All Rights Reserved.
License: Apache-2.0
Comment: Quoting AUTHORS:
 # This is the official list of benchmark authors for copyright purposes.
 Antonio Marcedone 
 Google LLC
 Internet Security Research Group
 Vishal Kuo 

The idea is that from a legal perspective, the copyright notices and
keywords 'FSFAP' and 'Apache-2.0' with full text copy of the license is
sufficient documentation.  However, for reasons like proper attribution
and having more background information, it is useful to say something
more than what's legally required, including properly quoting the
relevant files.  I think the Comment: section makes for a better place
than License: fields for this.

Does anyone have other advice related to gnulib's debian/copyright file?

I have yet to fully get a grip on how this file should best reflect
reality for a complex package like gnulib, but will try to do my best to
resolve lintian complaints and keep it accurate and maintainable.

/Simon


signature.asc
Description: PGP signature

Re: Validating tarballs against git repositories

[Simon Josefsson]

Colin Watson  writes:

> On Mon, Apr 01, 2024 at 11:33:06AM +0200, Simon Josefsson wrote:
>> Running ./bootstrap in a tarball may lead to different results than the
>> maintainer running ./bootstrap in pristine git.  It is the same problem
>> as running 'autoreconf -fvi' in a tarball does not necessarily lead to
>> the same result as the maintainer running 'autoreconf -fvi' from
>> pristine git.  The different is what is pulled in from the system
>> environment.  Neither tool was designed to be run from within a tarball,
>> so this is just bad practice that never worked reliable and without a
>> lot of complexity it will likely not become reliable either.
>
> The practice of running "autoreconf -fi" or similar via dh-autoreconf
> has worked extremely well at scale in Debian.  I'm sure there are
> complex edge cases where it's caused problems, but it's far from being a
> disaster area.

Agreed.  I'm saying it doesn't fix the problem that I perceive that some
people appear to believe, i.e., that running 'autoreconf -fi' solves the
re-bootrapping problem.  Only some files get re-generated, such as the
./configure script, which is good, but not all files.  It wouldn't have
solved the xz case: build-to-host.m4 wouldn't have been re-generated.

With a *-src.tar.gz approach [1], the build-to-host.m4 file shouldn't
even be part of the tarball.  That would be easier to detect during an
audit of list of files compared to git repository, rather than waiting
for code review of file content (which usually only happens when
debugging some real-world problem).

[1] 
https://blog.josefsson.org/2024/04/01/towards-reproducible-minimal-source-code-tarballs-please-welcome-src-tar-gz/

> I don't think running ./bootstrap can be generalized as easily as
> running autoreconf can, and it's definitely going to be tough to apply
> to all packages that use gnulib; but I think the blanket statement that
> it's bad practice is painting with too broad a brush.  For the packages
> where I've applied it so far (most of which I'm upstream for,
> admittedly), it's fine.

I'm not saying autoreconf -fi is bad practice, I'm saying it is
incomplete and leads to a feeling of having solved the re-bootstrapping
problem that isn't backed by facts.

>> I have suggested before that upstream's (myself included) should publish
>> PGP-signed *-src.tar.gz tarballs that contain the entire pristine git
>> checkout including submodules,
>
> A while back I contributed support to Gnulib's bootstrap script to allow
> pinning particular commits without using submodules.  I would recommend
> this mode; submodules have very strange UI.

I never liked git submodules generally, so I would be happy to work on
getting that to be supported -- do you have pointers for earlier works
here?

What is necessary, I think, is having something like this in
bootstrap.conf:

gnulib_commit_id = 123abc567...

and it would then use the external git repository pointed to by
--gnulib-refdir and locate that commit, and extract the gnulib files
from that gnulib commit.  And refuse to continue if it can't find that
particular commit.

This is essentially the same as a git submodule -- encoding the gnulib
commit to use in the project's own git history -- but without the bad
git submodule user experience.

I use different approaches to gnulib in projects. In OATH Toolkit I
still put all gnulib-generated content in git because running
./bootstrap otherwise used to take several minutes.  In most projects I
have given up and use git submodules.  In some I rely on running
gnulib-tool from git, and the exact gnulib git commit to use is only
whatever I happened to have checked out on my development machine.

>> *.po translations,
>
> As I noted in a comment on your blog, I think there is a case to be made
> for .po files being committed to upstream git, and I'm not fond of the
> practice of pulling them in only at bootstrap time (although I can
> understand why that's come to be popular as a result of limited
> maintainer time).  I have several reasons to believe this:

Those are all good arguments, but it still feels backwards to put these
files into git.  It felt so good to externalize all the translation
churn outside of my git (or then, CVS...) repositories many years ago.

I would prefer to maintain a po/SHA256SUMS in git and continue to
download translations but have some mechanism to refuse to continue if
the hashes differ.

/Simon


signature.asc
Description: PGP signature

Re: Validating tarballs against git repositories

[Simon Josefsson]

"G. Branden Robinson"  writes:

> At 2024-03-31T22:32:49+, Stefano Rivera wrote:
>> Upstreams would probably prefer that we used git repositories
>> *directly* as source artifacts, but that comes with a whole other can
>> of worms...
>
> Speaking from my upstream groff perspective, I wouldn't _prefer_ that.
>
> The distribution archives get build-testing on a much wider variety of
> systems, thanks to people on the groff@ and platform-testers@gnu mailing
> lists that help out when a release candidate is announced.  They have
> access to platforms more exotic that I and a few other bleeding-edge
> HEAD mavens do.  This practice tangibly improved the quality of the
> groff 1.23.0 release, especially on surviving proprietary Unix systems.
>
> Building from the repo, or using the bootstrap script--which Colin
> Watson just today ensured will be in future distribution archives--is
> fine.[1]  I'm glad some people build the project that way.  But I think
> that procedure serves an audience that is distinguishable in some ways.

Running ./bootstrap in a tarball may lead to different results than the
maintainer running ./bootstrap in pristine git.  It is the same problem
as running 'autoreconf -fvi' in a tarball does not necessarily lead to
the same result as the maintainer running 'autoreconf -fvi' from
pristine git.  The different is what is pulled in from the system
environment.  Neither tool was designed to be run from within a tarball,
so this is just bad practice that never worked reliable and without a
lot of complexity it will likely not become reliable either.

I have suggested before that upstream's (myself included) should publish
PGP-signed *-src.tar.gz tarballs that contain the entire pristine git
checkout including submodules, *.po translations, and whatever else is
required to actually build the project that is normally pulled in from
external places (autoconf archive macros?).  This *-src.tar.gz tarball
should be possible to ./bootstrap and that would be the intended way to
build it for people who care about vendored files.  Thoughts?  Perhaps I
should formalize this proposal a bit more.

/Simon

> Regards,
> Branden
>
> [1] 
> https://git.savannah.gnu.org/cgit/groff.git/commit/?id=822fef56e9ab7cbe69337b045f6f20e32e25f566
>


signature.asc
Description: PGP signature

Re: Git and SHA1 collisions

[Simon Josefsson]

Gioele Barabucci  writes:

> But pulling a successful collision attack is not a trivial task. For
> instance, the xz attacker did not have all that was required to carry
> it out (for example they had no direct access to the git
> servers... yet).

Is that necessary?  It seems that if you have push access, you can push
a colliding commit.  Does GitLab on Salsa verify (and reject?) colliding
commit ids a'la SHA1-CD?  Would the tag2upload git server do that?

/Simon


signature.asc
Description: PGP signature

Re: xz backdoor

[Simon Josefsson]

Bastian Blank  writes:

> On Sun, Mar 31, 2024 at 12:05:54PM +0500, Andrey Rakhmatullin wrote:
>> On Sat, Mar 30, 2024 at 11:22:33PM -0300, Santiago Ruano Rincón wrote:
>> > As others have said, the best solution is to relay on HSW for handling
>> > the cryptographic material.
>> Aren't these answers to different questions?
>> Not all attacks are about stealing the key or using it to sign unintended
>> things.
>
> Also a HSM does only allow to control access to the cryptographic
> material.  But it asserts no control over what is actually signed.

Transparency techniques are better suited to solve that problem: make
sure that you don't trust a signature before verifying that the
signature was publicly logged together with its artifacts, so that they
can be independently audited and analyzed eventually.  Preferrably even
verify that the package artifacts build reproducible, but that takes
more resources.  Right now Debian trust signatures at face value which
is fragile.  The WebPKI world -- which is populated by untrustworthy
private key signers -- has moved in that direction, and it does improve
things.

/Simon


signature.asc
Description: PGP signature

Re: Validating tarballs against git repositories

[Simon Josefsson]

Russ Allbery  writes:

> Simon Josefsson  writes:
>> Sean Whitton  writes:
>
>>> We did some analysis on the SHA1 vulnerabilities and determined that
>>> they did not meaningfully affect dgit & tag2upload's design.
>
>> Can you share that analysis?  As far as I understand, it is possible for
>> a malicious actor to create a git repository with the same commit id as
>> HEAD, with different historic commits and tree content.  I thought a
>> signed tag is merely a signed reference to a particular commit id.  If
>> that commit id is a SHA1 reference, that opens up for ambiguity given
>> recent (well, 2019) results on SHA1.  Of course, I may be wrong in any
>> of the chain, so would appreciate explanation of how this doesn't work.
>
> I believe you're talking about two different things.  I think Sean is
> talking about preimage resistance, which assumes that the known-good
> repository is trusted, and I believe Simon is talking about manufactured
> collisions where the attacker controls both the good and the bad
> repository.

Right.  I think the latter describes the xz scenario: someone could have
pushed a maliciously crafted commit with a SHA1 collision commit id, so
there are two different git repositories with that commit id, and a
signed git tag on that commit id authenticates both trees, opening up
for uncertainty about what was intended to be used.  Unless I'm missing
some detail of how git signed tag verification works that would catch
this.

> The dgit and tag2upload design probably (I'd have to think about it some
> more, ideally while bouncing the problem off of someone else, because I've
> recycled those brain cells for other things) only needs preimage
> resistance, but the general case of a malicious upstream may be vulnerable
> to manufactured collisions.

It is not completely clear to me: How about if some malicious person
pushed a commit to salsa, asked a DD to "please review this repository
and sign a tag to make the upload"?  The DD would presumably sign a
commit id that authenticate two different git trees, one with the
exploit and one without it.

/Simon


signature.asc
Description: PGP signature

Re: Validating tarballs against git repositories

[Simon Josefsson]

Jonathan Carter  writes:

> On 2024/03/30 11:05, Simon Josefsson wrote:
>>> 1. Move towards allowing, and then favoring, git-tags over source tarballs
>>
>> Some people have suggested this before -- and I have considered adopting
>> that approach myself, but one thing that is often overlooked is that
>> building from git usually increase the Build-Depends quite a lot
>> compared to building from tarball
>
> How in the world do you jump to that conclusion?

By comparing the set of tools required to build from git with the tools
installed by Build-Depends* for common projects.  I'm thinking of
projects like coreutils, wget, libidn2, gnutls, gzip, etc.

/Simon


signature.asc
Description: PGP signature

Re: Validating tarballs against git repositories

[Simon Josefsson]

Sean Whitton  writes:

> Hello,
>
> On Sat 30 Mar 2024 at 12:19pm +01, Simon Josefsson wrote:
>
>> Relying on signed git tags is not reliable because git is primarily
>> SHA1-based which in 2019 cost $45K to do a collission attack for.
>
> We did some analysis on the SHA1 vulnerabilities and determined that
> they did not meaningfully affect dgit & tag2upload's design.

Can you share that analysis?  As far as I understand, it is possible for
a malicious actor to create a git repository with the same commit id as
HEAD, with different historic commits and tree content.  I thought a
signed tag is merely a signed reference to a particular commit id.  If
that commit id is a SHA1 reference, that opens up for ambiguity given
recent (well, 2019) results on SHA1.  Of course, I may be wrong in any
of the chain, so would appreciate explanation of how this doesn't work.

/Simon


signature.asc
Description: PGP signature

Re: Validating tarballs against git repositories

[Simon Josefsson]

Gioele Barabucci  writes:

> Just as an example, bootstrapping coreutils currently requires
> bootstrapping at least 68 other packages, including libx11-6 [1]. If 
> coreutils supported  [2], the transitive closure of its
> Build-Depends would be reduced to 20 packages, most of which in 
> build-essential.
>
> [1]
> https://buildd.debian.org/status/fetch.php?pkg=coreutils&arch=amd64&ver=9.4-3.1&stamp=1710441056&raw=1
> [2] https://bugs.debian.org/1057136

Coreutils in Debian uses upstream tarballs and does not do a full
bootstrap build.  It does autoreconf instead of ./bootstrap.  So the
dependencies above is not the entire bootstrapping story to build
coreutils from git compared to building from tarballs.

It would help if upstreams would publish PGP-signed 'git-archive'-style
tarballs, including content from git submodules in them.

Relying on signed git tags is not reliable because git is primarily
SHA1-based which in 2019 cost $45K to do a collission attack for.

/Simon


signature.asc
Description: PGP signature

Re: Validating tarballs against git repositories

[Simon Josefsson]

Antonio Russo  writes:

> 1. Move towards allowing, and then favoring, git-tags over source tarballs

Some people have suggested this before -- and I have considered adopting
that approach myself, but one thing that is often overlooked is that
building from git usually increase the Build-Depends quite a lot
compared to building from tarball, and that will more likely trigger
cyclic dependencies.  People that do bootstrapping for new platforms or
cross-platform dislike such added dependency.

One response to that may be "sorry, our concerns for supply chain
security trumps your desire for easier building" but so far I believe
the approach has been to compromise a little on supply chain side (i.e.,
building from tarballs) and compromise a little on the
bootstrap/crossbuild smoothness (e.g., adding nodoc or nocheck targets).

Moving that needle isn't all that trivial, although I think I'm moving
myself to a preference that we really need to build everything from
source code and preferrably not even including non-source code files
because they may dormant and activated later on a'la the xz attack.

An old irk of mine is that people seems to believe that running
'autoreconf -fi' is intended or supposed to combat problems related to
this: autoreconf was never designed for that purpose, nor does it
achieve it realiably.  Many distributions have adopted a preference to
do run 'autoreconf' to "re-bootstrap" a project from source code.  This
misses a lot of generated files, and sometimes generate incorrect (and
possibly harmful) newly generated files.  For example:
https://gitlab.com/libidn/libidn2/-/issues/108

/Simon


signature.asc
Description: PGP signature

Re: Transparency into private keys of Debian

[Simon Josefsson]

Hans-Christoph Steiner  writes:

>> In business, such things are confirmed (often badly) by independent
>> audit. For a volunteer-driven community effort, we have to rely on
>> everyone to exercise their best judgement in these sorts of matters.
>
> Debian could also get independent, professional audits.  I think it
> would be a good use of the Debian pot of money, for example.  Or
> someone could submit a proposal to get Debian audited.  I'll be either
> Open Tech Fund or NLnet would do it:
>
> https://www.opentech.fund/labs/red-team-lab/
>
> Open Tech Fund already funds Tails, which is based on Debian.

That would be useful for the important keys like the apt release keys,
and would set an example for others to follow.  If there are things to
improve, it would be better if we know about them than allowing
attackers to find out on their own.  For DD keys, as Jemery noticed, I
don't think it is useful: uses of DD keys leave a quite auditable flow
already.

/Simon


signature.asc
Description: PGP signature

Re: Transparency into private keys of Debian

[Simon Josefsson]

> > I've looked at Sigstore, it looks nice.  It seems to be architected
> > for use 
> > cases that assume highly reliable and unblocked single domains. 
> > That's a 
> > showstopper for us.  Also, the official client app is 100% JVM code
> > right now 
> > (Java+Kotlin), so integrating Go binaries would really be an option
> > of last 
> > resort.  Its almost a no go for many reasons.
> 
> Agreed -- having a C, perl or python client for Sigstore or Sigsum
> would be nice.  I'll bring this up with them.

I should have checked before sending the previous email, there is this
client:

https://git.glasklar.is/sigsum/core/sigsum-py

I suppose for apt python is more relevant, and if the sigsum proof is
included alongside apt metadata, it would allow offline verification
inside apt.  If apt doesn't support this natively, the proof would have
to be distributed through some other channel, but that is relatively
easy to do as a Proof-of-Concept (i.e., how apt-sigstore +
debdistcanary already works via gitlab) but scaling it to decentralized
etc remains to be resolved, if relevant.

/Simon

> 
> /Simon
> 
> > 
> > Hans
> > 
> > > 
> > > /Simon
> > > 
> > > > And publishing the jurisdictions could be enough info to
> > > > deanonymize
> > > > the key holder, e.g. if it is Germany, then there are many DDs
> > > > there.
> > > > If it is Iceland, then govs can easily find who to target.
> > > > 
> > > > .hc
> > > > 
> > > > > Hi
> > > > > I'm exploring how to defend against an attacker who can
> > > > > create
> > > > > valid
> > > > > signatures for cryptographic private keys (e.g., PGP) that
> > > > > users need to
> > > > > trust when using an operating system such as Debian.  A
> > > > > signature like
> > > > > that can be used in a targetted attacks against one victim.
> > > > > For example, apt does not have any protection against this
> > > > > threat
> > > > > scenario, and often unprotected ftp:// or http:// transports
> > > > > are used,
> > > > > which are easy to man-in-the-middle.  Even with https://
> > > > > there
> > > > > is a
> > > > > large number of mirror sites out there that can replace
> > > > > content
> > > > > you get.
> > > > > There is a risk that use of a compromised trusted apt PGP key
> > > > > will not
> > > > > be noticed.  Attackers are also in a good position to deny
> > > > > themselves
> > > > > out of their actions if they are careful.
> > > > > Part of my effort is to inventory all trusted private keys
> > > > > that
> > > > > a
> > > > > distribution needs to manage on their own, or depend on that
> > > > > are managed
> > > > > by others, and gain some insight how these keys are handled.
> > > > > Does the Debian project, or any members, publish information
> > > > > on
> > > > > these
> > > > > topics?  Has this been discussed before?
> > > > > Some questions that I think would be useful to answer
> > > > > include:
> > > > > 1) List of relevant private keys, held by the organization,
> > > > > its
> > > > > members,
> > > > >     or someone else.  As far as I can tell from Debian, the
> > > > > list includes
> > > > >     the following PGP trust anchors:
> > > > >    B8B8 0B5B 623E AB6A D877  5C45 B7C5 D7D6 3509 47F8
> > > > >    05AB 9034 0C0C 5E79 7F44  A8C8 254C F3B5 AEC0 A8F0
> > > > >    4D64 FEC1 19C2 0290 67D6  E791 F8D2 585B 8783 D481
> > > > >    1F89 983E 0081 FDE0 18F3  CC96 73A4 F27B 8DD4 7936
> > > > >    AC53 0D52 0F2F 3269 F5E9  8313 A484 4904 4AAD 5C5D
> > > > >    A428 5295 FC7B 1A81 6000  62A9 605C 66F0 0D6C 9793
> > > > >    80D1 5823 B7FD 1561 F9F7  BCDD DC30 D7C2 3CBB ABEE
> > > > >    5E61 B217 265D A980 7A23  C5FF 4DFA B270 CAA9 6DFA
> > > > >    6D33 866E DD8F FA41 C014  3AED DCC9 EFBF 77E1 1517
> > > > >     Compromising any single key on this list leads to
> > > > > trouble.
> > > > > However I
> > > > >     don't think this list is complete.  What other keys are
> > > > > there?
> > > > >     I believe there are keys used to sign some component of
> > > > > the
> > > > > boot
> > > > >     phase, compare the 'shim-signed' and 'grub-efi-amd64-
> > > > > signed'
> > > > >     packages.  What private keys held by Debian are involved
> > > > > here?  What
> > > > >     keys held by others are involved?  What other similar
> > > > > packages
> > > > >     exists?
> > > > > 2) For each private key, information about its management and
> > > > > lifecycle.
> > > > >     Relevant questions include:
> > > > >   a) How was the key generated?  By whom?  On what hardware? 
> > > > > What
> > > > >  software?  In what environment?  What legal jurisdiction
> > > > > apply to
> > > > >  people involved?
> > > > >   b) How is the key stored and protected during its
> > > > > lifetime? 
> > > > > What
> > > > > media
> > > > >  is used?  Who control the physical storage of the key? 
> > > > > How are they
> > > > >  stored and transported?  What jurisdiction?
> > > > >   c) Under what policy is the key used?  What

Re: Transparency into private keys of Debian

[Simon Josefsson]

tis 2024-02-06 klockan 16:50 +0100 skrev Hans-Christoph Steiner:
> 
> 
> Simon Josefsson:
> > Hans-Christoph Steiner  writes:
> > 
> > > Thanks for digging in here, its very important work!  I'd be
> > > happy to
> > > contribute where I can.  I'm a DD and a core contributor to F-
> > > Droid,
> > > where we wrestle with basically the same issues.  So we've
> > > thought a
> > > lot about these kinds of things, but definitely do not have all
> > > the
> > > answers.  Since F-Droid started much later than Debian, we were
> > > able
> > > to build in some nice protections from the beginning, like
> > > requiring
> > > HTTPS.  We've also made the decision to stick with a single
> > > jurisdiction for the singing keys, even though it is not the best
> > > one
> > > for our needs.  We'd of course love to move them to the best
> > > jurisdiction but that is actually quite a bit of work, so we have
> > > to
> > > stay grounded in reality.
> > > 
> > > For me the hard part of all this is how to be transparent as
> > > possible
> > > without putting a giant target on the heads of the people who
> > > control
> > > the keys.  I think it is clear that publishing private key usage
> > > information is safe, like this key signed these things at these
> > > times.
> > > Publishing all the details about how the key was generated could
> > > help
> > > those who want to attack it more than those who rely on it.  But
> > > that
> > > kind of information would be good to share internally to the
> > > project
> > > at the very least.
> > 
> > Good aspect -- and indeed this is a trade-off, and thanks for
> > caring
> > about these issues for f-droid!  It seems that if you would verify
> > signatures via a transparency log, you would reduce the risk on the
> > people controlling the keys?  Then you (and they) could feel more
> > comfortable publishing information how your process work. That
> > would be
> > valuable to publish (even de-personalized or generalized) as a best
> > practices approach.  The reason is that anyone stealing the keys
> > from
> > these persons would ALSO have to upload signatures of whatever they
> > wanted to use these keys for into the transparency log, which would
> > provide evidence to what they did, so they may be less likely to
> > follow
> > through on their plans.
> 
> Yes, I agree, and am familiar with this architecture.  The hard part
> is the 
> availability of the transparency log needs to be as good as the
> signed index's 
> availability, otherwise attackers can just block the transparency log
> to stop 
> the whole process.  The official F-Droid client does a lot of things 
> automatically in order to find a working mirror.  And we're expanding
> on that. 
> I have yet to see a transparency log set up for decentralized,
> flexible 
> distribution.  F-Droid's mirror handling will automatically choose
> from:
> 
> * f-droid.org
> * Any of the official mirrors
> * Any mirrors that the user has added locally
> * Files hosted on IPFS
> * Mirrors on local storage (SD cards, thumb drives, etc)
> 
> We're expanding to:
> * Mirrors on cloud file storage like Nextcloud, Google Drive, etc.
> * Signed JSON on public code distribution platforms and CDNs

I believe sigsum allows for offline verification of the inclusion
proof.  Just include the transparency checksum proof with the app or
meta-data around the app.  Not sure about Sigstore.

> (As a side note: I would like to see Debian/apt adopt this approach,
> at least 
> optionally.  The key part is automatic operation and dead simple
> setup, e.g. for 
> non-technical users.)

+1

> > What would be involved is to 1) during signing of artifacts, also
> > sign
> > and upload into Sigstore/Sigsum, and 2) during verification in the
> > f-droid app, also verify that the signature has been committed to
> > the
> > Sigstore/Sigsum logs.  Both projects have clients written in Go
> > which
> > should work on Android, but the rest of the details are sketchy to
> > me.
> > I'm happy to continue discuss and help with design if you are
> > interested, to understand what the limitations of your environments
> > are
> > and how to resolve them.
> 
> I've looked at Sigstore, it looks nice.  It seems to be architected
> for use 
> cases that assume highly reliable and unblocked single domains. 
> That's a 
>

Re: Transparency into private keys of Debian

[Simon Josefsson]

Stephan Verbücheln  writes:

> II. Typical Debian case
>
> 1. Debian developer signs source tarballs and upload them
> 2. The signature only has to be secure until the code lands in the FTP
> 3. Debian builds the binary packages
> 4. Debian creates Release files with hashes of the packages
> 5. The Release file is signed by Debian APT keys
> 6. The signature has to be secure until the next Release file
> 7. Security updates have a separate Release file with expiration time
>
> This strategy effectively prevents attackers from injecting outdated
> programs with known vulnerabilities into the updates. Even mirrors
> cannot do that. TLS for mirrors is not required for integrity and
> authenticity.

Thanks for summarizing.  I'm not concerned with injecting outdated
programs -- I'm concerned with injecting malware indirectly signed by
the Debian APT keys, and want to explore ways to mitigate against those
attacks.

Understanding the private key lifecycle is one way to increase
confidence in what the cost of those attacks are -- right now all we can
say is that the cost of forging signatures for Debian APT keys is likely
higher than $0 and is likely lower than, let's say, $100B to pay someone
to get the keys or inject code for malicious remote signing.  Tightening
these bounds, using some rational methodology, gives us knowledge about
safety.

Using transparency logging of the signatures created by the private keys
is another way to allow detection when this occur, which also increase
confidence in the overall security, and reduce the risk with the private
keys (since they are longer useful for deniable attacks).

It may be that publishing more details about the private key lifecycle
is not a good idea unless we have measurements to protect ourselves, but
at least establishing that fact would be an improvement, and an
incentive to invest in these improvements.

/Simon


signature.asc
Description: PGP signature

Re: Transparency into private keys of Debian

[Simon Josefsson]

Bill Allombert  writes:

> On Mon, Feb 05, 2024 at 08:49:09AM +0100, Simon Josefsson wrote:
>> Bill Allombert  writes:
>> 
>> > Le Thu, Feb 01, 2024 at 10:38:03AM +0100, Simon Josefsson a écrit :
>> >> Hi
>> >> 
>> >> I'm exploring how to defend against an attacker who can create valid
>> >> signatures for cryptographic private keys (e.g., PGP) that users need to
>> >> trust when using an operating system such as Debian.  A signature like
>> >> that can be used in a targetted attacks against one victim.
>> >> 
>> >> For example, apt does not have any protection against this threat
>> >> scenario, 
>> >
>> > Is not apt-key a protection ?
>> 
>> No, the current implementation protects against missing and/or invalid
>> signatures.  Compare how in the WebPKI world some CA issued a valid
>> *.google.com certificate, and how that (and other incidents) lead to
>> setup of Certificate Transparency, which helps mitigate these issues.
>
> The difference is that with apt-key, the list of valid public keys is stored 
> on
> the user system (in /etc/apt/trusted.gpg.d/), not a list of root certificates,
> and that the users are notified when the keys are updated, which is not the
> case with CA.  Nobody can generate a new signature that will be accepted by
> apt-key out of the box.

Right, there are differences, but I believe the underlying problem is
the same: if someone mis-use private keys I trust, or is able to forge
valid signatures somehow (e.g., crypto attack), apt-key wouldn't notice
but in the WebPKI world there are mechanisms to mitigate this via
Certificate Transparency.

It is a subjective choice to care about this attack scenario, and some
may regard it as out of scope, but others can believe it is in scope and
that improvements are warranted.

/Simon


signature.asc
Description: PGP signature

Re: Transparency into private keys of Debian

[Simon Josefsson]

Hans-Christoph Steiner  writes:

> Thanks for digging in here, its very important work!  I'd be happy to
> contribute where I can.  I'm a DD and a core contributor to F-Droid,
> where we wrestle with basically the same issues.  So we've thought a
> lot about these kinds of things, but definitely do not have all the
> answers.  Since F-Droid started much later than Debian, we were able
> to build in some nice protections from the beginning, like requiring
> HTTPS.  We've also made the decision to stick with a single
> jurisdiction for the singing keys, even though it is not the best one
> for our needs.  We'd of course love to move them to the best
> jurisdiction but that is actually quite a bit of work, so we have to
> stay grounded in reality.
>
> For me the hard part of all this is how to be transparent as possible
> without putting a giant target on the heads of the people who control
> the keys.  I think it is clear that publishing private key usage
> information is safe, like this key signed these things at these times.
> Publishing all the details about how the key was generated could help
> those who want to attack it more than those who rely on it.  But that
> kind of information would be good to share internally to the project
> at the very least.

Good aspect -- and indeed this is a trade-off, and thanks for caring
about these issues for f-droid!  It seems that if you would verify
signatures via a transparency log, you would reduce the risk on the
people controlling the keys?  Then you (and they) could feel more
comfortable publishing information how your process work. That would be
valuable to publish (even de-personalized or generalized) as a best
practices approach.  The reason is that anyone stealing the keys from
these persons would ALSO have to upload signatures of whatever they
wanted to use these keys for into the transparency log, which would
provide evidence to what they did, so they may be less likely to follow
through on their plans.

What would be involved is to 1) during signing of artifacts, also sign
and upload into Sigstore/Sigsum, and 2) during verification in the
f-droid app, also verify that the signature has been committed to the
Sigstore/Sigsum logs.  Both projects have clients written in Go which
should work on Android, but the rest of the details are sketchy to me.
I'm happy to continue discuss and help with design if you are
interested, to understand what the limitations of your environments are
and how to resolve them.

/Simon

> And publishing the jurisdictions could be enough info to deanonymize
> the key holder, e.g. if it is Germany, then there are many DDs there.
> If it is Iceland, then govs can easily find who to target.
>
> .hc
>
>> Hi
>> I'm exploring how to defend against an attacker who can create valid
>> signatures for cryptographic private keys (e.g., PGP) that users need to
>> trust when using an operating system such as Debian.  A signature like
>> that can be used in a targetted attacks against one victim.
>> For example, apt does not have any protection against this threat
>> scenario, and often unprotected ftp:// or http:// transports are used,
>> which are easy to man-in-the-middle.  Even with https:// there is a
>> large number of mirror sites out there that can replace content you get.
>> There is a risk that use of a compromised trusted apt PGP key will not
>> be noticed.  Attackers are also in a good position to deny themselves
>> out of their actions if they are careful.
>> Part of my effort is to inventory all trusted private keys that a
>> distribution needs to manage on their own, or depend on that are managed
>> by others, and gain some insight how these keys are handled.
>> Does the Debian project, or any members, publish information on
>> these
>> topics?  Has this been discussed before?
>> Some questions that I think would be useful to answer include:
>> 1) List of relevant private keys, held by the organization, its
>> members,
>>or someone else.  As far as I can tell from Debian, the list includes
>>the following PGP trust anchors:
>>   B8B8 0B5B 623E AB6A D877  5C45 B7C5 D7D6 3509 47F8
>>   05AB 9034 0C0C 5E79 7F44  A8C8 254C F3B5 AEC0 A8F0
>>   4D64 FEC1 19C2 0290 67D6  E791 F8D2 585B 8783 D481
>>   1F89 983E 0081 FDE0 18F3  CC96 73A4 F27B 8DD4 7936
>>   AC53 0D52 0F2F 3269 F5E9  8313 A484 4904 4AAD 5C5D
>>   A428 5295 FC7B 1A81 6000  62A9 605C 66F0 0D6C 9793
>>   80D1 5823 B7FD 1561 F9F7  BCDD DC30 D7C2 3CBB ABEE
>>   5E61 B217 265D A980 7A23  C5FF 4DFA B270 CAA9 6DFA
>>   6D33 866E DD8F FA41 C014  3AED DCC9 EFBF 77E1 1517
>>Compromising any single key on this list leads to trouble.
>> However I
>>don't think this list is complete.  What other keys are there?
>>I believe there are keys used to sign some component of the boot
>>phase, compare the 'shim-signed' and 'grub-efi-amd64-signed'
>>packages.  What private keys held by Debian are involved here?  What
>>keys held

Re: Transparency into private keys of Debian

[Simon Josefsson]

Bill Allombert  writes:

> Le Thu, Feb 01, 2024 at 10:38:03AM +0100, Simon Josefsson a écrit :
>> Hi
>> 
>> I'm exploring how to defend against an attacker who can create valid
>> signatures for cryptographic private keys (e.g., PGP) that users need to
>> trust when using an operating system such as Debian.  A signature like
>> that can be used in a targetted attacks against one victim.
>> 
>> For example, apt does not have any protection against this threat
>> scenario, 
>
> Is not apt-key a protection ?

No, the current implementation protects against missing and/or invalid
signatures.  Compare how in the WebPKI world some CA issued a valid
*.google.com certificate, and how that (and other incidents) lead to
setup of Certificate Transparency, which helps mitigate these issues.
It is possible to implement similar features for the relevant private
keys used to sign Debian too; Sigstore and Sigsum are two publicly
available projects.

/Simon


signature.asc
Description: PGP signature

Transparency into private keys of Debian

[Simon Josefsson]

Hi

I'm exploring how to defend against an attacker who can create valid
signatures for cryptographic private keys (e.g., PGP) that users need to
trust when using an operating system such as Debian.  A signature like
that can be used in a targetted attacks against one victim.

For example, apt does not have any protection against this threat
scenario, and often unprotected ftp:// or http:// transports are used,
which are easy to man-in-the-middle.  Even with https:// there is a
large number of mirror sites out there that can replace content you get.
There is a risk that use of a compromised trusted apt PGP key will not
be noticed.  Attackers are also in a good position to deny themselves
out of their actions if they are careful.

Part of my effort is to inventory all trusted private keys that a
distribution needs to manage on their own, or depend on that are managed
by others, and gain some insight how these keys are handled.

Does the Debian project, or any members, publish information on these
topics?  Has this been discussed before?

Some questions that I think would be useful to answer include:

1) List of relevant private keys, held by the organization, its members,
   or someone else.  As far as I can tell from Debian, the list includes
   the following PGP trust anchors:

  B8B8 0B5B 623E AB6A D877  5C45 B7C5 D7D6 3509 47F8
  05AB 9034 0C0C 5E79 7F44  A8C8 254C F3B5 AEC0 A8F0
  4D64 FEC1 19C2 0290 67D6  E791 F8D2 585B 8783 D481
  1F89 983E 0081 FDE0 18F3  CC96 73A4 F27B 8DD4 7936
  AC53 0D52 0F2F 3269 F5E9  8313 A484 4904 4AAD 5C5D
  A428 5295 FC7B 1A81 6000  62A9 605C 66F0 0D6C 9793
  80D1 5823 B7FD 1561 F9F7  BCDD DC30 D7C2 3CBB ABEE
  5E61 B217 265D A980 7A23  C5FF 4DFA B270 CAA9 6DFA
  6D33 866E DD8F FA41 C014  3AED DCC9 EFBF 77E1 1517

   Compromising any single key on this list leads to trouble.  However I
   don't think this list is complete.  What other keys are there?

   I believe there are keys used to sign some component of the boot
   phase, compare the 'shim-signed' and 'grub-efi-amd64-signed'
   packages.  What private keys held by Debian are involved here?  What
   keys held by others are involved?  What other similar packages
   exists?

2) For each private key, information about its management and lifecycle.
   Relevant questions include:

 a) How was the key generated?  By whom?  On what hardware?  What
software?  In what environment?  What legal jurisdiction apply to
people involved?

 b) How is the key stored and protected during its lifetime?  What media
is used?  Who control the physical storage of the key?  How are they
stored and transported?  What jurisdiction?

 c) Under what policy is the key used?  What should it sign?  Who
authorize the signing?  What hardware and software is used?  What
jurisdiction?

 d) For externally held keys, what are the legal terms we use the keys
under?  What insight into key transparency questions do we have?
What of those can we make public?  How do they restrict what we are
allowed to do?

3) Which less visible private keys are indirectly trusted by users of
   the distribution?  For example, all DD PGP keys are indirectly
   trusted since they are permitted to make uploads into the archive.
   Host keys used to authorized access to sensitive systems may also be
   relevant.  I'm primarily thinking SSH private keys to a system that
   may have online access to a private key signing oracle.  Indirectly,
   questions about authentication protocols and authorization methods
   are relevant.

To the extent that symmetric shared secrets (rather than asymmetric
public/private keys) are involved, the same question applies.

Other aspects worth exploring?

I understand commercial distributions have different incentives related
to making this information public.  They have a commercial interest to
defend, and has usually entered various legal arrangements that create
obstacles related to releasing information.  As we've seen with the
WebPKI CA business, that model does not inspire trust and leads to
systematic failures.  More transparent approaches like Certificate
Transparency and LetsEncrypt evolved as a consequence.

I believe that Debian is in a good position to improve things and, if
there is interest, could lead the way by documenting a better approach,
and describe how to deal with these concerns in a more transparent way
than what we do today.

Thoughts?

/Simon


signature.asc
Description: PGP signature

Bug#1061446: ITP: cosign -- Code signing and transparency for containers and binaries

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 

* Package name: cosign
  Version : 2.2.2-1
  Upstream Author : The Sigstore Authors
* URL : https://github.com/sigstore/cosign
* License : Apache-2.0
  Programming Lang: Go
  Description : Code signing and transparency for containers and binaries

 Signing OCI containers (and other artifacts) using Sigstore
 (https://sigstore.dev/)!
 .
 Go Report Card
 (https://goreportcard.com/report/github.com/sigstore/cosign) e2e-tests
 (https://github.com/sigstore/cosign/actions/workflows/e2e-tests.yml) CII
 Best Practices
 (https://bestpractices.coreinfrastructure.org/projects/5715) OpenSSF
 Scorecard
 (https://api.securityscorecards.dev/projects/github.com/sigstore/cosign)
 .
 Cosign aims to make signatures **invisible infrastructure**.
 .
 Cosign supports:
 .
  * "Keyless signing" with the Sigstore public good Fulcio certificate
authority and Rekor transparency log (default)
  * Hardware and KMS signing
  * Signing with a cosign generated encrypted private/public keypair
  * Container Signing, Verification and Storage in an OCI registry.
  * Bring-your-own PKI
 .
 Info
 .
 Cosign is developed as part of the sigstore (https://sigstore.dev)
 project. We also use a slack channel (https://sigstore.slack.com)! Click
 here (https://join.slack.com/t/sigstore/shared_invite/zt-mhs55zh0-
 XmY3bcfWn4XEyMqUUutbUQ) for the invite link.
 .
 Installation
 .
 For Homebrew, Arch, Nix, GitHub Action, and Kubernetes installs see the
 installation docs
 (https://docs.sigstore.dev/system_config/installation/).
 .
 For Linux and macOS binaries see the GitHub release assets
 (https://github.com/sigstore/cosign/releases/latest).
 .
 :rotating_light: If you are downloading releases of cosign from our GCS
 bucket - please see more information on the July 31, 2023 deprecation
 notice (https://blog.sigstore.dev/cosign-releases-bucket-deprecation/)
 :rotating_light:
 .
 Developer Installation
 .
 If you have Go 1.19+, you can setup a development environment:
 .
   $ git clone https://github.com/sigstore/cosign
   $ cd cosign
   $ go install ./cmd/cosign
   $ $(go env GOPATH)/bin/cosign
 .
 Contributing
 .
 If you are interested in contributing to cosign, please read the
 contributing documentation (/CONTRIBUTING.md).
 .
 Dockerfile
 .
 Here is how to install and use cosign inside a Dockerfile through the
 gcr.io/projectsigstore/cosign image:
 .
   FROM gcr.io/projectsigstore/cosign:v1.13.0 as cosign-bin
 .
   # Source: https://github.com/chainguard-images/static
   FROM cgr.dev/chainguard/static:latest
   COPY --from=cosign-bin /ko-app/cosign /usr/local/bin/cosign
   ENTRYPOINT [ "cosign" ]
 .
 Quick Start
 .
 This shows how to:
 .
  * sign a container image with the default "keyless signing" method (see
KEYLESS.md (/KEYLESS.md))
  * verify the container image
 .
 Sign a container and store the signature in the registry
 .
 Note that you should always sign images based on their digest
 (@sha256:...) rather than a tag (:latest) because otherwise you might
 sign something you didn't intend to!
 .
cosign sign $IMAGE
 .
   Generating ephemeral keys...
   Retrieving signed certificate...
 .
Note that there may be personally identifiable information associated
 with this signed artifact.
This may include the email address associated with the account with
 which you authenticate.
This information will be used for signing this artifact and will be
 stored in public transparency logs and cannot be removed later.
 .
   By typing 'y', you attest that you grant (or have permission to grant)
 and agree to have this information stored permanently in transparency
 logs.
   Are you sure you would like to continue? [y/N] y
   Your browser will now be opened to:
 .
 
https://oauth2.sigstore.dev/auth/auth?access_type=online&client_id=sigstore&code_challenge=OrXitVKUZm2lEWHVt1oQWR4HZvn0rSlKhLcltglYxCY&code_challenge_method=S256&nonce=2KvOWeTFxYfxyzHtssvlIXmY6Jk&redirect_uri=http%3A%2F%2Flocalhost%3A57102%2Fauth%2Fcallback&response_type=code&scope=openid+email&state=2KvOWfbQJ1caqScgjwibzK2qJmb
   Successfully verified SCT...
   tlog entry created with index: 12086900
   Pushing signature to: $IMAGE
 .
 Cosign will prompt you to authenticate via OIDC, where you'll sign in
 with your email address. Under the hood, cosign will request a code
 signing certificate from the Fulcio certificate authority. The subject
 of the certificate will match the email address you logged in with.
 Cosign will then store the signature and certificate in the Rekor
 transparency log, and upload the signature to the OCI registry alongside
 the image you're signing.
 .
 Verify a container
 .
 To verify the image, you'll need to pass in the expected certificate
 issuer and certificate subject via the --certificate-identity and --
 certificate-oidc-issuer flags:
 .
   co

Re: Proposal for how to deal with Go/Rust/etc security bugs

[Simon Josefsson]

Luca Boccassi  writes:

> On Wed, 24 Jan 2024 at 13:34, Simon Josefsson  wrote:
>>
>> Luca Boccassi  writes:
>>
>> >> Having reflected a bit, and learned through my own experience and
>> >> others' insights [1] that Go Build-Depends are not transitive, I'd like
>> >> to update my proposal on how to handle a security bug in any Go/Rust/etc
>> >> package and the resulting package rebuilds:
>> >
>> > There's always option B: recognize that the Rust/Go ecosystems are not
>> > designed to be compatible with the Linux distributions model
>>
>> Definitely - that's roughly the model we have today, right?  So no
>> action needed to preserve status quo of option B.
>>
>> I want to explore if there is a possibility to change status quo, and
>> what would be required to do so.
>
> What's required is talking to the language ecosystem owners and
> convince them to support a stable ABI and dynamic linking, and in
> general to care about the distribution use case. Otherwise it's just
> an unwinnable uphill battle that consumes a ton of scarce resources
> (developers time), and is simply hopeless.

One could equally well make the argument that distributors should care
about the Go/Rust ecosystems, and make whatever changes needed in order
to support them.  Those changes are what I'm trying to explore here.

Speaking as a C person (I know little about Go/Rust), getting stable
ABIs, dynamic linking and security upgrades right is not simple, and
we've been working on that for 20+ years consuming plenty of human
resources on the way.

/Simon


signature.asc
Description: PGP signature

Re: Proposal for how to deal with Go/Rust/etc security bugs

[Simon Josefsson]

Luca Boccassi  writes:

>> Having reflected a bit, and learned through my own experience and
>> others' insights [1] that Go Build-Depends are not transitive, I'd like
>> to update my proposal on how to handle a security bug in any Go/Rust/etc
>> package and the resulting package rebuilds:
>
> There's always option B: recognize that the Rust/Go ecosystems are not
> designed to be compatible with the Linux distributions model

Definitely - that's roughly the model we have today, right?  So no
action needed to preserve status quo of option B.

I want to explore if there is a possibility to change status quo, and
what would be required to do so.

Given how often gnulib is vendored for C code in Debian, and other
similar examples, I don't think of this problem as purely a Go/Rust
problem.  The parallel argument that we should not support coreutils,
sed, tar, gzip etc because they included vendored copies of gnulib code
is not reasonable.

> and are instead designed to be as convenient as possible for a
> _single_ application developer and its users - at the detriment of
> everybody else - and for large corporations that ship a handful of
> applications with swathes of engineers that can manage the churn, and
> it is not made nor intended to scale to ~60k packages or whatever
> number we have today in unstable. And simply stop attempting to merge
> these two incompatible ecosystems against their will, at the very
> least until and unless they reach feature and stability parity with
> the C/C++ ecosystems - stable API/ABI and dynamic linking by default.

Go seems to have supported shared libraries since around ca 2015:

https://go.dev/talks/2015/state-of-go-may.slide#13
https://docs.google.com/document/d/1nr-TQHw_er6GOQRsF6T43GGhFDelrAP0NqSS_00RgZQ/edit

Does anyone know of a shared library in a Debian package written in Go?
I've only encountered the vendored approach to ship Go libraries.

> There are many ways to ship applications today that are much better
> suited for these models, like Flatpak, where the maintenance burden is
> shifted onto those who choose to opt in to such ecosystems.

Or simply 'go install ...', it works remarkably well.

However, this is orthogonal to how we support the Go code that is in
Debian already.

/Simon


signature.asc
Description: PGP signature

Proposal for how to deal with Go/Rust/etc security bugs (was: Re: Limited security support for Go/Rust? Re ssh3)

[Simon Josefsson]

Simon Josefsson  writes:

>> > My naive approach on how to fix a security problem in package X
>> > which is
>> > statically embedded into other packages A, B, C, ... would be to
>> > rebuild
>> > the transitive closure of all packages that Build-Depends on X and
>> > publish a security update for all those packages.
...
> I realized that there is one problem with my approach: consider if
> package A was built via Build-Depends package B of version X and that
> later package B is upgraded to X+1 in Debian.  Then if a security
> problem happens in B we need to rebuild A. It may be that package A no
> longer builds due to some incompatibility between version X and X+1 of
> B.  This would not be noticed until a full rebuild of an archive is
> done, or when the security teams wants to rebuild the transitive
> closure of the Build-Depends graph for a package.

Having reflected a bit, and learned through my own experience and
others' insights [1] that Go Build-Depends are not transitive, I'd like
to update my proposal on how to handle a security bug in any Go/Rust/etc
package and the resulting package rebuilds:

  To fix a security problem in package X, which is used during build
  (through statical linking, vendoring, or some other mechanism) to
  build package A, B, C, ... you need to rebuild all packages that
  Build-Depends on X (lets call this step 1) and all packages that
  Build-Depends on any of the packages that will be rebuilt during step
  1.  This rebuild process is iterated until no more packages remains to
  be rebuild, and all packages have been rebuilt using newly rebuild
  packages.  If there are cyclical dependencies (which unfortunately are
  common), you have to loop until all packages have been rebuilt with a
  clean dependency chain, and detect the loop and stop rebuilding.  If
  there are FTBFS errors during the rebuilds, this will have to be
  patched too.

Yes, for a low-level Go package (e.g., golang-golang-x-net-dev), this
will mean rebuilding almost all of the Go packages in Debian and publish
them in a security advisory.

This algorithm can be optimized (i.e., reduce the number of packages to
publish in an advisory) by either of:

1) using information from Built-Using: (which was not designed for
   this purpose, so this is fragile) or *.buildinfo.

2) by dropping all 'Architecture: all' packages that does not embedd
   the buggy code.

The last optimization 2) would reduce the number of Go packages to
publish significantly, as it would drop most golang-*-dev packages.  I
think this actually makes this process feasible in practice, as there
are relatively few binary packages written in Go.

This method applies to non-Go/Rust too, if there are such security
problems and reverse build chains.

I think all of this (except maybe the optimization 2) which requires
code comparisons) can be automated in a GitLab pipeline for higher
confidence of the result.

/Simon

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=806481#58


signature.asc
Description: PGP signature

Bug#1061153: ITP: sigsum-go -- tools for public and transparent logging of signed checksums

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 

* Package name: sigsum-go
  Version : 1.7.1-1
  Upstream Author : The Sigsum Project Authors
* URL : https://git.glasklar.is/sigsum/core/sigsum-go
* License : BSD-2-Clause
  Programming Lang: Go
  Description : tools for public and transparent logging of signed checksums

 The goal of Sigsum is to provide building blocks that can be used to
 enforce public logging of signed checksums.
 .
 This package contains the sigsum-key, sigsum-submit, sigsum-verify,
 and sigsum-monitor command line tools.

I hope to maintain this package as part of Debian Go Packaging Team:

https://salsa.debian.org/go-team/packages/sigsum-go

/Simon


signature.asc
Description: PGP signature

Re: Limited security support for Go/Rust? Re ssh3

[Simon Josefsson]

Adam Borowski  writes:

> On Mon, Jan 15, 2024 at 10:17:17AM +0100, Bastian Blank wrote:
>> On Sun, Jan 14, 2024 at 04:24:57PM +0100, Simon Josefsson wrote:
>> > Isn't that what the text refers to?  Vendoring and static linking are
>> > two examples of the same problem that the security team may encounter.
>> 
>> We accept vendoring of autoconf/automake/gnulib distro wide.
>
> We _did_ accept that in the past, but these days you get smacked with a RC
> bug for not building from source.

What do you mean?  Can you show me _any_ package in Debian that
re-bootstrap itself using gnulib during package build?  If by source you
mean the source code in gnulib, not the vendored version shipping with
many packages.

If a security bug is found in gnulib code, one approach to fix it would
be to patch the Debian gnulib package, and then automatically rebuild
all packages that uses that gnulib function.

I believe it is rare for packages in Debian to re-bootstrap itself from
gnulib sources.  Look at coreutils, sed, grep, tar, wget, libidn2, etc,
none of them do that.  Certainly not a RC bug.

The situation now when a security bug in gnulib is found, you will have
to patch all packages using that code manually per-package.

>> The vendoring of gnulib, well, is old and maybe you could
>> show that it is a problem in the sources that have it, aka they don't
>> handle security fixes and at the same time don't change the library.
>
> Gnulib has not been useful for ages, thus packages still shipping vendored
> copies are harmless -- functions that gnulib was meant to provide
> implementation for were missing on ancient unices like HP-UX or SCO that
> are long dead by now.  A glance at recent commits in gnulib shows a lot of
> retrocomputing names: Windows, OS/2, MacOS 10.5, AIX, on hardware of that
> level of recency.  It's not used for new ports: the most recent reference
> to riscv in commit messages is from _2018_.

I think you underestimate how often gnulib is used, and how well it
works on modern platforms, and how much gnulib code is used that's not
in glibc or other system shared libraries.  I assume coreutils, sed,
tar, gzip etc were important bootstrapping packages for riscv, and they
all build fine thanks in large extent due to gnulib being written in a
architecture independent way.

/Simon


signature.asc
Description: PGP signature

Bug#1061050: ITP: golang-github-common-nighthawk-go-figure -- Prints ASCII art from text

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 

* Package name: golang-github-common-nighthawk-go-figure
  Version : 0.0~git20210622.734e95f-1
  Upstream Author : Daniel Deutsch
* URL : https://github.com/common-nighthawk/go-figure
* License : Expat
  Programming Lang: Go
  Description : Prints ASCII art from text

 Go Figure prints beautiful ASCII art from text. It supports FIGlet
 (http://www.figlet.org/) files, and most of its features.
 .
 This package was inspired by the Ruby gem artii
 (https://github.com/miketierney/artii), but built from scratch and with
 a different feature set.

I hope to maintain this package as part of Debian Go Packaging Team:

https://salsa.debian.org/go-team/packages/golang-github-common-nighthawk-go-figure

/Simon


signature.asc
Description: PGP signature

Re: Limited security support for Go/Rust? Re ssh3

[Simon Josefsson]

Bastian Blank  writes:

> On Tue, Jan 16, 2024 at 10:59:30AM +0100, Simon Josefsson wrote:
>> Rebuilding a bit more than what is strictly needed sounds fine as a
>> first solution to me.
>
> Building maybe.  But how do you want to publish them?  The security
> archive is not made to handle that.

What is the limitation?  Is it human time involved to rebuild and QA
packages?  Or the technical infrastructure related to publish security
fixes?  Or the hosting infrastructure to deliver the package upgrades
via security.debian.org?  Or something else?

>> My naive approach on how to fix a security problem in package X which is
>> statically embedded into other packages A, B, C, ... would be to rebuild
>> the transitive closure of all packages that Build-Depends on X and
>> publish a security update for all those packages.
>
> So if a fix to the net/tls module of go shows up (happens from time to
> time), all go packages needs to be rebuilt?

All Go packages that are affected by the security problem of net/tls,
yes.  That seems to be how the Go eco-system want things to be, the
static linking nature is by design.  Whether that is a good or bad
design can be discussed, but it seems intentional and won't go away.

My main point is that we have this situation in Debian already.  Maybe
vendoring via gnulib is a better example than config.h or statical
linking though.

Having being more exposed to the packaging aspects of Go projects in
Debian recently, I am more sympathetic towards being conservative about
offering security claims for these packages though.  I wouldn't envy
anyone trying to resolve a security vulnerability for a Go package in
bookworm with many reverse dependencies.  OTOH, I don't think it is
sustainable to ignore security vulnerabilities in software that we ship.
We just have to find a technical solution to solve whatever problems
prevent us from being able to do that.

/Simon


signature.asc
Description: PGP signature

Re: Limited security support for Go/Rust? Re ssh3

[Simon Josefsson]

"IOhannes m zmölnig (Debian GNU|Linux)"  writes:

> On 1/16/24 13:56, Jérémy Lal wrote:
>>>
>>> As Built-Using is for license compliance only, no?
>>>
>>> See
>>>
>>> https://www.debian.org/doc/debian-policy/ch-relationships.html#additional-source-packages-used-to-build-the-binary-built-using
>> Indeed, thanks for the link.
>> 
>
> it seems that many people think that "Built-Using" can be used to
> express static linking (including yours truly, even though i *know*
> that it is meant for license compliance only).
>
> which makes me wonder: probably we should have an additional field
> that expresses such static linking (and therefore would trigger a
> rebuild when the dependency changes).
> or we could finally accept that many¹ people would just use
> "Built-Using" for this anyhow, and explicitly allow such use.

Would that be better or worse than making *.buildinfo files more
generally available and required?

Buildinfo files appears to have some traction already, and it seems like
they could help address the same problem.

Unfortunately *.buildinfo still seems hard to access reliably and their
integrity aren't protected by the archive-wide InRelease signature, if I
understand correctly.

/Simon


signature.asc
Description: PGP signature

Re: Limited security support for Go/Rust? Re ssh3

[Simon Josefsson]

tis 2024-01-16 klockan 11:22 +0100 skrev Jérémy Lal:
> 
> 
> Le mar. 16 janv. 2024 à 11:00, Simon Josefsson 
> a écrit :
> > Paul Wise  writes:
> > 
> > > On Mon, 2024-01-15 at 10:17 +0100, Bastian Blank wrote:
> > > 
> > > > I asked for practical solutions, not theoretical ones.  We
> > > > don't have a
> > > > suitable way to rebuild all packages just because right now.
> > > 
> > > There are some ideas on the static linking wiki page:
> > > 
> > > https://wiki.debian.org/StaticLinking
> > > 
> > > Probably the most practical solution for today would be to use a
> > > build
> > > info database to find out which builds had installed binary
> > > packages
> > > containing insecure statically linkable files of any kind, then
> > > rebuild
> > > the source packages that were affected. There is a 2019 demo
> > > here:
> > > 
> > > https://salsa.debian.org/bremner/builtin-pho/-/blob/master/demos/needs-rebuild.sh
> > > https://www.cs.unb.ca/~bremner//blog/posts/builtin-pho/
> > > 
> > > This may mean rebuilding more packages than were really needed,
> > > but a more exact method would require full tracing of input data
> > > to
> > > output data during builds being added to all toolchains, which
> > > seems
> > > like a much longer term project than buildinfo based rebuilds.
> > 
> > Rebuilding a bit more than what is strictly needed sounds fine as a
> > first solution to me.
> > 
> > My naive approach on how to fix a security problem in package X
> > which is
> > statically embedded into other packages A, B, C, ... would be to
> > rebuild
> > the transitive closure of all packages that Build-Depends on X and
> > publish a security update for all those packages.
> > 
> > What is the problem with that approach to handle security problems
> > in a
> > Go package for trixie?
> > 
> > I'm sure the number of packages to rebuild could be reduced in
> > various
> > clever ways, but until we have tooling to automate that, a naive
> > but
> > costly approach seems feasible, unless i'm missing something.
> > 
> > How large is the gap between tracing buildinfo information and
> > simply
> > relying on Build-Depends?
> > 
> > Isn't the gap between using Build-Depends and the buildinfo-
> > approach
> > only concerning the always-assumed-to-be-installed packages like
> > libc or
> > /bin/sh which I never seems to recall if they are what build-
> > essential
> > installs, or what the policy manual says it should do, or what
> > 'Essential: yes' implies, or 'Priority: required' implies, etc. 
> > For Go
> > packages I don't think they are relevant in practice.
> > 
> 
> 
> I naively believed that golang-* packages expressed those
> dependencies with "Built-Using".

True, but I was thinking of a solution that would not be Go-specific.

I realized that there is one problem with my approach: consider if
package A was built via Build-Depends package B of version X and that
later package B is upgraded to X+1 in Debian.  Then if a security
problem happens in B we need to rebuild A. It may be that package A no
longer builds due to some incompatibility between version X and X+1 of
B.  This would not be noticed until a full rebuild of an archive is
done, or when the security teams wants to rebuild the transitive
closure of the Build-Depends graph for a package.

I still think this is something we just need to be prepared to handle,
by patching packages to fix the build problem in whatever way is
appropriate.  It will require some more patching to really fix the
security problem, and allow packages to be rebuilt with the new
version.

/Simon



signature.asc
Description: This is a digitally signed message part

Re: Limited security support for Go/Rust? Re ssh3

[Simon Josefsson]

Paul Wise  writes:

> On Mon, 2024-01-15 at 10:17 +0100, Bastian Blank wrote:
>
>> I asked for practical solutions, not theoretical ones.  We don't have a
>> suitable way to rebuild all packages just because right now.
>
> There are some ideas on the static linking wiki page:
>
> https://wiki.debian.org/StaticLinking
>
> Probably the most practical solution for today would be to use a build
> info database to find out which builds had installed binary packages
> containing insecure statically linkable files of any kind, then rebuild
> the source packages that were affected. There is a 2019 demo here:
>
> https://salsa.debian.org/bremner/builtin-pho/-/blob/master/demos/needs-rebuild.sh
> https://www.cs.unb.ca/~bremner//blog/posts/builtin-pho/
>
> This may mean rebuilding more packages than were really needed,
> but a more exact method would require full tracing of input data to
> output data during builds being added to all toolchains, which seems
> like a much longer term project than buildinfo based rebuilds.

Rebuilding a bit more than what is strictly needed sounds fine as a
first solution to me.

My naive approach on how to fix a security problem in package X which is
statically embedded into other packages A, B, C, ... would be to rebuild
the transitive closure of all packages that Build-Depends on X and
publish a security update for all those packages.

What is the problem with that approach to handle security problems in a
Go package for trixie?

I'm sure the number of packages to rebuild could be reduced in various
clever ways, but until we have tooling to automate that, a naive but
costly approach seems feasible, unless i'm missing something.

How large is the gap between tracing buildinfo information and simply
relying on Build-Depends?

Isn't the gap between using Build-Depends and the buildinfo-approach
only concerning the always-assumed-to-be-installed packages like libc or
/bin/sh which I never seems to recall if they are what build-essential
installs, or what the policy manual says it should do, or what
'Essential: yes' implies, or 'Priority: required' implies, etc.  For Go
packages I don't think they are relevant in practice.

/Simon


signature.asc
Description: PGP signature

Bug#1060853: ITP: golang-github-sigstore-protobuf-specs -- Sigstore Protocol Buffer code (library)

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 

* Package name: golang-github-sigstore-protobuf-specs
  Version : 0.2.1-1
  Upstream Author : sigstore
* URL : https://github.com/sigstore/protobuf-specs
* License : Apache-2.0
  Programming Lang: Go
  Description : Sigstore Protocol Buffer code (library)

 This repository holds protobuf specifications for Sigstore messages.

I hope to maintain this package as part of Debian Go Packaging Team:

https://salsa.debian.org/go-team/packages/golang-github-sigstore-protobuf-specs

/Simon


signature.asc
Description: PGP signature

Bug#1060852: ITP: golang-bitbucket-creachadair-shell -- implements basic shell command-line splitting for Go (library)

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 

* Package name: golang-bitbucket-creachadair-shell
  Version : 0.0.8-1
  Upstream Author : Michael J. Fromberger
* URL : https://bitbucket.org/creachadair/shell/
* License : BSD-3-Clause
  Programming Lang: Go
  Description : implements basic shell command-line splitting for Go 
(library)

 Provides supports for splitting and joining of shell command strings.
 .
 The Split function divides a string into whitespace-separated fields,
 respecting single and double quotation marks as defined by the Shell Command
 Language section of IEEE Std 1003.1 2013.  The Quote function quotes
 characters that would otherwise be subject to shell evaluation, and the Join
 function concatenates quoted strings with spaces between them.

I hope to maintain this package as part of Debian Go Packaging Team:

https://salsa.debian.org/go-team/packages/golang-bitbucket-creachadair-shell/

/Simon


signature.asc
Description: PGP signature

Bug#1060842: ITP: trillian -- A transparent, highly scalable and cryptographically verifiable data store

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 

* Package name: trillian
  Version : 1.6.0-1
  Upstream Author : Google
* URL : https://github.com/google/trillian
* License : Apache-2.0
  Programming Lang: Go
  Description : A transparent, highly scalable and cryptographically 
verifiable data store

 Trillian is an implementation of the concepts described in the
 Verifiable Data Structures (/docs/papers/VerifiableDataStructures.pdf)
 white paper, which in turn is an extension and generalisation of the
 ideas which underpin Certificate Transparency (https://certificate-
 transparency.org).
 .
 Trillian implements a Merkle tree
 (https://en.wikipedia.org/wiki/Merkle_tree) whose contents are served
 from a data storage layer, to allow scalability to extremely large
 trees.  On top of this Merkle tree, Trillian provides the following:
 .
  * An append-only **Log** mode, analogous to the original
Certificate Transparency (https://certificate-transparency.org) logs.
In
this mode, the Merkle tree is effectively filled up from the left,
giving a
*dense* Merkle tree.
 .
 Certificate Transparency (CT) (https://tools.ietf.org/html/rfc6962) is
 the most well-known and widely deployed transparency application, and an
 implementation of CT as a Trillian personality is available in the
 certificate-transparency-go repo (https://github.com/google/certificate-
 transparency-go/blob/master/trillian).

I hope to maintain this package as part of Debian Go Packaging Team:

https://salsa.debian.org/go-team/packages/trillian

/Simon


signature.asc
Description: PGP signature

Bug#1060841: ITP: golang-github-transparency-dev-merkle -- create and manipulate Merkle trees in Go (library)

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 

* Package name: golang-github-transparency-dev-merkle
  Version : 0.0.2-1
  Upstream Author : Pavel Kalinnikov, Al Cutter, Martin Hutchinson, M Hickford, 
et al
* URL : https://github.com/transparency-dev/merkle
* License : Apache-2.0
  Programming Lang: Go
  Description : create and manipulate Merkle trees in Go (library)

 Provides Go code to help create and manipulate Merkle trees, as well as
 constructing and verifying various types of proof.
 .
 This is the data structure which is used by projects such as Trillian
 (https://github.com/google/trillian) to provide verifiable logs
 (https://transparency.dev/verifiable-data-structures/#verifiable-log).

I hope to maintain this package as part of Debian Go Packaging Team:

https://salsa.debian.org/go-team/packages/golang-github-transparency-dev-merkle

/Simon


signature.asc
Description: PGP signature

Bug#1060840: ITP: golang-k8s-sigs-release-utils -- utilities for kubernetes Go release engineering (library)

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 

* Package name: golang-k8s-sigs-release-utils
  Version : 0.7.7-1
  Upstream Author : Kubernetes SIGs
* URL : https://github.com/kubernetes-sigs/release-utils
* License : Apache-2.0
  Programming Lang: Go
  Description : utilities for kubernetes Go release engineering (library)

 Tiny utilities for use by the Release Engineering subproject and
 kubernetes/release (https://github.com/kubernetes/release/).

I hope to maintain this package as part of Debian Go Packaging Team:

https://salsa.debian.org/go-team/packages/golang-k8s-sigs-release-utils

/Simon


signature.asc
Description: PGP signature

Bug#1060839: ITP: golang-github-adamkorcz-go-fuzz-headers-1 -- helper functions for Go fuzzing (library)

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 

* Package name: golang-github-adamkorcz-go-fuzz-headers-1
  Version : 0.0~git20230919.8b5d3ce-1
  Upstream Author : Adam Korcz 
* URL : https://github.com/AdamKorcz/go-fuzz-headers-1
* License : Apache-2.0
  Programming Lang: Go
  Description : helper functions for Go fuzzing (library)

 Various helper functions for go fuzzing. It is mostly used in combination
 with go-fuzz (https://github.com/dvyukov/go-fuzz), but compatibility with
 fuzzing in the standard library will also be supported. Any coverage guided
 fuzzing engine that provides an array or slice of bytes can be used with
 go-fuzz-headers.
 .
 go-fuzz-headers' approach to fuzzing structs is strongly inspired by
 gofuzz (https://github.com/google/gofuzz).

I hope to maintain this package as part of Debian Go Packaging Team:

https://salsa.debian.org/go-team/packages/golang-github-adamkorcz-go-fuzz-headers-1/

/Simon


signature.asc
Description: PGP signature

Bug#1060836: ITP: golang-github-cavaliergopher-rpm -- A Go implementation of the RPM file format

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 

* Package name: golang-github-cavaliergopher-rpm
  Version : 1.2.0-1
  Upstream Author : Ryan Armstrong, et al
* URL : https://github.com/cavaliergopher/rpm
* License : BSD-3-clause
  Programming Lang: Go
  Description : Go implementation of the RPM file format (library)

 This implements the rpm package file format as a Go library
 .
 See the package documentation
 (https://pkg.go.dev/github.com/cavaliergopher/rpm) to get started.

I hope to maintain this package as part of Debian Go Packaging Team:

https://salsa.debian.org/go-team/packages/golang-github-cavaliergopher-rpm

/Simon


signature.asc
Description: PGP signature

Bug#1060834: ITP: golang-github-veraison-go-cose -- go library for CBOR Object Signing and Encryption (COSE)

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 

* Package name: golang-github-veraison-go-cose
  Version : 1.2.1-1
  Upstream Author : Veraison
* URL : https://github.com/veraison/go-cose
* License : MPL-2.0
  Programming Lang: Go
  Description : go library for CBOR Object Signing and Encryption (COSE)

 A golang library for the COSE specification
 (https://datatracker.ietf.org/doc/rfc9052/)

I hope to maintain this package as part of Debian Go Packaging Team:

https://salsa.debian.org/go-team/packages/golang-github-veraison-go-cose

/Simon


signature.asc
Description: PGP signature

Bug#1060820: ITP: golang-github-cyberphone-json-canonicalization -- JSON Canonicalization Scheme (JCS) (Go library)

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 

* Package name: golang-github-cyberphone-json-canonicalization
  Version : 0.0~git20220623.57a0ce2-1
  Upstream Author : Anders Rundgren
* URL : https://github.com/cyberphone/json-canonicalization
* License : Apache-2.0
  Programming Lang: Go
  Description : JSON Canonicalization Scheme (JCS) (Go library)

 Cryptographic operations like hashing and signing depend on that the
 target data does not change during serialization, transport, or parsing.
 By applying the rules defined by JCS (JSON Canonicalization Scheme),
 data provided in the JSON [RFC8259
 (https://tools.ietf.org/html/rfc8259)] format can be exchanged "as is",
 while still being subject to secure cryptographic operations. JCS
 achieves this by building on the serialization formats for JSON
 primitives as defined by ECMAScript [ES (https://ecma-
 international.org/ecma-262/)], constraining JSON data to the I-JSON
 [RFC7493 (https://tools.ietf.org/html//rfc7493)] subset, and through a
 platform independent property sorting scheme.
 .
 Public RFC: (https://tools.ietf.org/html/rfc8785)
 .
 The JSON Canonicalization Scheme concept in a nutshell:
 .
  * Serialization of primitive JSON data types using methods compatible
with ECMAScript's JSON.stringify()
  * Lexicographic sorting of JSON Object properties in a *recursive*
process
  * JSON Array data is also subject to canonicalization, *but element
order remains untouched*

I hope to maintain this package as part of Debian Go Packaging Team:

https://salsa.debian.org/go-team/packages/golang-github-cyberphone-json-canonicalization

/Simon


signature.asc
Description: PGP signature

Bug#1060819: ITP: golang-github-zeebo-errs -- errs is a package for making errors friendly and easy

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 

* Package name: golang-github-zeebo-errs
  Version : 1.3.0-1
  Upstream Author : Jeff Wendling
* URL : https://github.com/zeebo/errs
* License : Expat
  Programming Lang: Go
  Description : errs is a Go library for making errors friendly and easy

 errs is a package for making errors friendly and easy.  Errors come
 with a stack trace that is only printed when a "+" character is used in
 the format string. This should retain the benefits of being able to
 diagnose where and why errors happen, without all of the noise of
 printing a stack trace in every situation.

I hope to maintain this package as part of Debian Go Packaging Team:

https://salsa.debian.org/go-team/packages/golang-github-zeebo-errs/

/Simon


signature.asc
Description: PGP signature

Bug#1060818: ITP: in-toto-golang -- framework for software supply chain integrity

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 

* Package name: in-toto-golang
  Version : 0.9.0-1
  Upstream Author : Aditya Sirish, Christian Rebischke, Lukas Pühringer, et al
* URL : https://github.com/in-toto/in-toto-golang
* License : Apache-2.0
  Programming Lang: Go
  Description : framework for software supply chain integrity

 Go implementation of in-toto

I hope to maintain this package as part of Debian Go Packaging Team:

https://salsa.debian.org/go-team/packages/in-toto-golang/

/Simon



signature.asc
Description: PGP signature

Bug#1060817: ITP: golang-github-spiffe-go-spiffe -- Golang library for SPIFFE support

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 

* Package name: golang-github-spiffe-go-spiffe
  Version : 2.1.6-1
  Upstream Author : Agustín Martínez Fayó, Andrew Harding, et al
* URL : https://github.com/spiffe/go-spiffe
* License : Apache-2.0
  Programming Lang: Go
  Description : Golang library for SPIFFE support

 This library is a convenient Go library for working with SPIFFE
 (https://spiffe.io/).
 .
 It leverages the SPIFFE Workload API
 (https://github.com/spiffe/spiffe/blob/main/standards/SPIFFE_Workload_API.md),
 providing high level functionality that includes:
 .
  * Establishing mutually authenticated TLS (**mTLS**) between workloads
powered by SPIFFE.
  * Obtaining and validating X509-SVIDs
(https://github.com/spiffe/spiffe/blob/main/standards/X509-SVID.md) and
JWT-SVIDs (https://github.com/spiffe/spiffe/blob/main/standards/JWT-
SVID.md).
  * Federating trust between trust domains using SPIFFE bundles

(https://github.com/spiffe/spiffe/blob/main/standards/SPIFFE_Trust_Domain_and_Bundle.md#3-
spiffe-bundles).
  * Bundle management.

I hope to maintain this package as part of Debian Go Packaging Team:

https://salsa.debian.org/go-team/packages/golang-github-spiffe-go-spiffe/

/Simon


signature.asc
Description: PGP signature

Bug#1060816: ITP: golang-github-shibumi-go-pathspec -- gitignore-style pathspec pattern matching in Go

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 

* Package name: golang-github-shibumi-go-pathspec
  Version : 1.3.0-1
  Upstream Author : Sander van Harmelen, Christian Rebischke
* URL : https://github.com/shibumi/go-pathspec
* License : Apache-2.0
  Programming Lang: Go
  Description : gitignore-style pathspec pattern matching in Go

 go-pathspec implements gitignore-style pattern matching for paths.

I hope to maintain this package as part of Debian Go Packaging Team:

https://salsa.debian.org/go-team/packages/golang-github-shibumi-go-pathspec/

/Simon


signature.asc
Description: PGP signature

Bug#1060815: ITP: relic -- digitally sign Linux/Java/Windows packages

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 

* Package name: relic
  Version : 7.6.1-1
  Upstream Author : SAS Institute, Inc.
* URL : https://github.com/sassoftware/relic
* License : Apache-2.0
  Programming Lang: Go
  Description : digitally sign Linux/Java/Windows packages

 relic is a multi-tool and server for package signing and working with
 hardware security modules (HSMs).
 .
 Package types
 .
  * RPM - RedHat packages
  * DEB - Debian packages
  * JAR - Java archives
  * EXE (PE/COFF) - Windows executable
  * MSI - Windows installer
  * appx, appxbundle - Windows universal application
  * CAB - Windows cabinet file
  * CAT - Windows security catalog
  * XAP - Silverlight and legacy Windows Phone applications
  * PS1, PS1XML, MOF, etc. - Microsoft Powershell scripts and modules
  * manifest, application - Microsoft ClickOnce manifest
  * VSIX - Visual Studio extension
  * Mach-O - macOS/iOS signed executables
  * DMG, PKG - macOS disk images / installer packages
  * APK - Android package
  * PGP - inline, detached or cleartext signature of data
 .
 Token types
 .
 relic can work with several types of token:
 .
  * pkcs11 - Industry standard PKCS#11 HSM interface using shared object
files
  * Cloud services - AWS, Azure and Google Cloud managed keys
  * scdaemon - The GnuPG scdaemon service can enable access to OpenPGP
cards (such as Yubikey NEO)
  * file - Private keys stored in a password-protected file
 .
 Features
 .
 Relic is primarily meant to operate as a signing server, allowing
 clients to authenticate with a TLS certificate and sign packages
 remotely. It can also be used as a standalone signing tool.
 .
 Other features include:
 .
  * Generating and importing keys in the token
  * Importing certificate chains from a PKCS#12 file
  * Creating X509 certificate signing requests (CSR) and self-signed
certificates
  * Limited X509 CA support -- signing CSRs and cross-signing certificates
  * Creating simple PGP public keys
  * RSA and ECDSA supported for all signature types
  * Verify signatures, certificate chains and timestamps on all supported
package types
  * Sending audit logs to an AMQP broker, with an optional sealing
signature
  * Save token PINs in the system keyring
 .
 Linux, Windows and MacOS are supported. Other platforms probably work as
 well.
 .
 relic is tested using libsofthsm2 and Gemalto SafeNet Network HSM (Luna
 SA).

I hope to maintain this package as part of Debian Go Packaging Team:

https://salsa.debian.org/go-team/packages/relic

/Simon


signature.asc
Description: PGP signature

Bug#1060813: ITP: golang-github-qur-ar -- Golang ar archive file library

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 

* Package name: golang-github-qur-ar
  Version : 0.0~git20130629.282534b-1
  Upstream Author : Blake Smith, Julian Phillips
* URL : https://github.com/qur/ar
* License : Expat
  Programming Lang: Go
  Description : Golang ar archive file library

 Golang ar (archive) file reader
 .
 This is a simple library for reading and writing ar
 (http://en.wikipedia.org/wiki/Ar_(Unix)) files in common format. It is
 influenced heavily in style and interface from the golang tar
 (http://golang.org/pkg/archive/tar/) package.

I hope to maintain this package as part of Debian Go Packaging Team:

https://salsa.debian.org/go-team/packages/golang-github-qur-ar

/Simon


signature.asc
Description: PGP signature

Bug#1060810: ITP: golang-github-sassoftware-go-rpmutils -- Golang implementation of parsing RPM packages

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 

* Package name: golang-github-sassoftware-go-rpmutils
  Version : 0.2.0-1
  Upstream Author : SAS Institute, Inc.
* URL : https://github.com/sassoftware/go-rpmutils
* License : Apache-2.0
  Programming Lang: Go
  Description : Golang implementation of parsing RPM packages

 go-rpmutils is a library written in go (http://golang.org) for parsing
 and extracting content from RPMs (http://www.rpm.org).
 .
 go-rpmutils provides a few interfaces for handling RPM packages. There is
 a highlevel Rpm struct that provides access to the RPM header and CPIO
 (https://en.wikipedia.org/wiki/Cpio) payload. The CPIO payload can be
 extracted to a filesystem location via the ExpandPayload function or
 through a Reader interface, similar to the tar implementation
 (https://golang.org/pkg/archive/tar/) in the go standard library.

I hope to maintain this package as part of Debian Go Packaging Team:

https://salsa.debian.org/go-team/packages/golang-github-sassoftware-go-rpmutils

/Simon


signature.asc
Description: PGP signature

Re: Limited security support for Go/Rust? Re ssh3

[Simon Josefsson]

Bastian Blank  writes:

> Hi Simon
>
> On Sun, Jan 14, 2024 at 10:47:18AM +0100, Simon Josefsson wrote:
>> As an analogy, consider the ./configure scripts that is generated by
>> autoconf during build of many packages.  The script typically generate
>> code that is put into config.h that is used (statically) during
>> compilation of the binaries that are shipped by Debian.
>
> Could you show an example, where there is actually code injcted in this
> stage?  And then, this is vendoring, not static linking.
>
>> You could also compare how the source-level reuse-library gnulib is used
>> by many essential packages (coreutils, grep, sed, awk, tar, etc), with
>> large code-reuse that influences the installed binaries.  A security
>> sensitive bug in gnulib would require rebuild of many packages.
>
> That is not static linking, this is vendoring.  And can you show that
> GNU utils don't fix security bugs on this vendored lib?
>
>> My suggestion is that we relax or remove the Go/Rust statement in future
>> release notes.
>
> No.  You described completely different circumstances.
>
> Or do you have a practical solution for the static linking problem, not
> the vendoring problem that you actually compared it against?

Right, these are slightly different technical problems, but as far as
the brief discussion in the release notes --
https://www.debian.org/releases/bookworm/amd64/release-notes/ch-information.en.html#golang-static-linking
-- I think the relevant aspect is identical: package X in Debian
contains an embedded copy of code whose corresponding source code is in
package Y in Debian, and fixing a security problem in Y will require
rebuilding X and the entire dependency chain between X and Y that carry
the code that ends up in X.

Isn't that what the text refers to?  Vendoring and static linking are
two examples of the same problem that the security team may encounter.
The problem with dependencies are more obvious for Go/Rust code but I
think we always have had that problem anyway.

As for solutions, isn't the solution to both vendoring or statical
linking the obvious one?  You will have to rebuild all packages that
contain the security vulnerability.

Maybe I'm missing how these two problems result in different problems
for the security team?

/Simon


signature.asc
Description: PGP signature

Re: Bug#1059618: ITP: ssh3 -- faster and rich secure shell using HTTP/3

[Simon Josefsson]

Bastian Blank  writes:

> On Fri, Dec 29, 2023 at 11:30:14AM +0100, Simon Josefsson wrote:
>> * Package name: ssh3
>
> This package name is clearly not acceptable.  SSH is a well known name
> and this project is completely unrelated to it.

Agreed.  Packagers have settled on using 'soh' for the name, see:

https://github.com/francoismichel/ssh3/issues/79
https://github.com/francoismichel/ssh3/pull/96

Once 0.1.5 is released, I will try to update the package to use the new
name.  It doesn't seem to collide with anything in Debian, as far as I
can tell.  It would be nice to have confirmation other distributions are
going to use the same name, but I think we have that in the discussion
above.  Thoughts?

Hyperbole in package descriptions are common, the current text is from
the upstream authors and I think it makes sense to copy that in the
package description.  If you can think of some improvement, consider
submitting a patch: https://salsa.debian.org/go-team/packages/ssh3

/Simon


signature.asc
Description: PGP signature

Limited security support for Go/Rust? Re ssh3

[Simon Josefsson]

Stephan Verbücheln  writes:

> On Sat, 30 Dec 2023 12:47:48 + Colin Watson 
> wrote:
>> I also feel that something security-critical like this that's
>> labelled by upstream as "still experimental" probably shouldn't
>> be in a Debian release.
>
> It is written in Go. The problem of Go library support in Debian should
> also be considered for a security-critical tool like this.
>
> https://www.debian.org/releases/bookworm/amd64/release-notes/ch-information.en.html#golang-static-linking

Interesting -- what is the current thinking about this problem?

The more I think about it, I think it seems unfair to categorize this as
a Go/Rust problem.

As an analogy, consider the ./configure scripts that is generated by
autoconf during build of many packages.  The script typically generate
code that is put into config.h that is used (statically) during
compilation of the binaries that are shipped by Debian.  Consider
further that these configure scripts would put security buggy into
config.h, coming from autoconf internally or some wildly used M4 macros.
How would the security team handle that situation?  Perhaps by patching
autoconf to fix the problem and rebuild all the packages that are
affected by that bug, and release them as a (potentially large) security
update.  This is a similar situation to the Go/Rust problem that the
link above describes.

Yes I know, historically config.h has been quite small, so the problem
hasn't been that significant, but have a look at a config.h from a
couple of large project today.  Compare an earlier large-scale-affecting
build-tool bug in automake --
https://lists.gnu.org/archive/html/automake/2012-07/msg00022.html --
which happened to have mild consequences because 'make distcheck' is
rarely run by most users or even during package builds, and the
generated code didn't end up in the installed binaries.  But it could
happen in a 'make all' or a 'make check' target too, or affect code that
actually influence the installed binaries.

You could also compare how the source-level reuse-library gnulib is used
by many essential packages (coreutils, grep, sed, awk, tar, etc), with
large code-reuse that influences the installed binaries.  A security
sensitive bug in gnulib would require rebuild of many packages.

My suggestion is that we relax or remove the Go/Rust statement in future
release notes.

/Simon


signature.asc
Description: PGP signature

Re: Bug#1059618: ITP: ssh3 -- faster and rich secure shell using HTTP/3

[Simon Josefsson]

Colin Watson  writes:

> On Sat, Dec 30, 2023 at 12:13:28AM +0100, Philipp Kern wrote:
>> On 29.12.23 11:30, Simon Josefsson wrote:
>> > SSH3 is a complete revisit of the SSH protocol, mapping its semantics on
>> > top of the HTTP mechanisms. In a nutshell, SSH3 uses QUIC+TLS1.3 for
>> > secure channel establishment and the HTTP Authorization mechanisms for
>> > user authentication. Among others, SSH3 allows the following
>> > improvements:
>> 
>> I feel like SSH3 is an unfortunate name. The program claims "SSH3 stands for
>> the concatenation of SSH and H3." - well sure, but you're also reusing the
>> name of an existing protocol and bump its version. ssh-h3?
>
> I agree - as the Debian OpenSSH maintainer, I'm concerned that this will
> cause a new source of user confusion because people will think "ah,
> ssh3, that must be better than ssh" (which indeed seems to have been a
> deliberate marketing choice by this project) and not realize that it's a
> largely incompatible thing.  Not to mention the way that it parses
> OpenSSH configuration files, which may work today but I doubt OpenSSH
> offers any guarantees that it won't make changes that will break this
> independent parser in future.

I share these concerns, so I'll delay the upload for now.  I'm hoping
upstream will rename the project to something less confusing.

> I also feel that something security-critical like this that's labelled
> by upstream as "still experimental" probably shouldn't be in a Debian
> release.  Maybe it should be kept in Debian experimental for the time
> being?

Sounds good if nothing happens on the naming front in the next
weeks/months.  Let's wait and see a bit.

One alternative that was suggested was to call the package something
else in Debian.  'golang-ssh3'?  'go-ssh3'?  Still somewhat problematic
as long as the 'ssh3' name is in there.

/Simon


signature.asc
Description: PGP signature

Re: Bug#1059618: ITP: ssh3 -- faster and rich secure shell using HTTP/3

[Simon Josefsson]

Packaging of SSH3 is available here:

https://salsa.debian.org/go-team/packages/ssh3
https://salsa.debian.org/jas/ssh3/

Thanks to the Salsa CI/CD pipeline there is an aptly repository
available for easy testing, if anyone would like to experiment or help.

Below you can find a snippet how you can test the SSH3 client and server
via Debian packages, for password and public key authentication, in a
safe container using podman.  I have only tested this on my laptop that
runs Trisquel, but should hopefully be portable.

I am delaying upload to Debian for a while to see if upstream reaches a
conclusion around naming.  I think the name 'ssh3' is unfortunate and
distracts from the effort. See:
.

/Simon

sudo apt install podman
podman run -it --hostname myhost.example --rm debian:unstable
cd
apt update
apt dist-upgrade -y
apt install -y ca-certificates
echo "deb [trusted=yes] 
https://salsa.debian.org/jas/ssh3/-/jobs/5094673/artifacts/raw/aptly unstable 
main" | tee /etc/apt/sources.list.d/ssh3.list
apt update
apt install -y ssh3

apt install -y ssl-cert # creates snakeoil key/cert

passwd # set a test password for 'root' e.g. 'foo'

ssh3-server -cert /etc/ssl/certs/ssl-cert-snakeoil.pem -key 
/etc/ssl/private/ssl-cert-snakeoil.key -enable-password-login -url-path /myurl 
-v &

ssh3 -v -insecure -use-password myhost.example/myurl
# type 'foo' at the prompt, and on successful connection type 'exit' to log out

apt install -y openssh-client # for ssh-keygen
ssh-keygen -t ed25519 -P "" -f /root/.ssh/id_ed25519
cat /root/.ssh/id_ed25519.pub > /root/.ssh3/authorized_identities
ssh3 -v -insecure -privkey /root/.ssh/id_ed25519 myhost.example/myurl
# on successful connection type 'exit' to log out


signature.asc
Description: PGP signature

Re: Bug#1059618: ITP: ssh3 -- faster and rich secure shell using HTTP/3

[Simon Josefsson]

Philipp Kern  writes:

> On 29.12.23 11:30, Simon Josefsson wrote:
>> Package: wnpp
>> Severity: wishlist
>> X-Debbugs-Cc: debian-devel@lists.debian.org, debian...@lists.debian.org
>> * Package name: ssh3
>>Version : 0.1.4
>>Upstream Contact: François Michel
>> * URL : https://github.com/francoismichel/ssh3
>> * License : Apache-2.0
>>Programming Lang: Go
>>Description : faster and rich secure shell using HTTP/3
>> SSH3 is a complete revisit of the SSH protocol, mapping its
>> semantics on
>> top of the HTTP mechanisms. In a nutshell, SSH3 uses QUIC+TLS1.3 for
>> secure channel establishment and the HTTP Authorization mechanisms for
>> user authentication. Among others, SSH3 allows the following
>> improvements:
>
> I feel like SSH3 is an unfortunate name. The program claims "SSH3
> stands for the concatenation of SSH and H3." - well sure, but you're
> also reusing the name of an existing protocol and bump its
> version. ssh-h3?
>
> Both the paper and the project are very new - so there should not be
> that many things referring to it yet.

I agree the name is unfortunate.  There are discussions with upstream in
https://github.com/francoismichel/ssh3/issues/79 and via emails.

I have packaging in https://salsa.debian.org/go-team/packages/ssh3 but I
will hold of uploading to NEW until some time has past to see if we
there will be a rename, and to give it more time for testing.  I have
managed to install the packages and start a server and make a client
connection to it.

/Simon


signature.asc
Description: PGP signature

Bug#1059620: ITP: golang-github-golang-jwt-jwt-v5 -- golang implementation of JSON Web Tokens (library)

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: Simon Josefsson 
X-Debbugs-CC: debian-devel@lists.debian.org, debian...@lists.debian.org

* Package name: golang-github-golang-jwt-jwt-v5
  Version : 5.2.0-1
  Upstream Author : golang-jwt maintainers, Dave Grijalva
* URL : https://github.com/golang-jwt/jwt
* License : Expat
  Programming Lang: Go
  Description : golang implementation of JSON Web Tokens (library)

I have working packaging of this here:

https://salsa.debian.org/jas/jwt-v5/

I hope to maintain it in the Debian Go Packaging group eventually.

This package appears to be required by the 'ssh3' package, and the
existing golang-github-golang-jwt-jwt package is stuck on the old ABI
and it looks like it will never upgrade, see:
https://tracker.debian.org/pkg/golang-github-golang-jwt-jwt

/Simon


signature.asc
Description: PGP signature

Bug#1059618: ITP: ssh3 -- faster and rich secure shell using HTTP/3

[Simon Josefsson]

Package: wnpp
Severity: wishlist
X-Debbugs-Cc: debian-devel@lists.debian.org, debian...@lists.debian.org

* Package name: ssh3
  Version : 0.1.4
  Upstream Contact: François Michel
* URL : https://github.com/francoismichel/ssh3
* License : Apache-2.0
  Programming Lang: Go
  Description : faster and rich secure shell using HTTP/3

SSH3 is a complete revisit of the SSH protocol, mapping its semantics on
top of the HTTP mechanisms. In a nutshell, SSH3 uses QUIC+TLS1.3 for
secure channel establishment and the HTTP Authorization mechanisms for
user authentication. Among others, SSH3 allows the following
improvements:

- Significantly faster session establishment

- New HTTP authentication methods such as OAuth 2.0 and OpenID Connect
  in addition to classical SSH authentication

- Robustness to port scanning attacks: your SSH3 server can be made
  invisible to other Internet users

- UDP port forwarding in addition to classical TCP port forwarding

- All the features allowed by the modern QUIC protocol: including
  connection migration (soon) and multipath connections

I hope this package can be maintained in the Debian Go Packaging Team.


signature.asc
Description: PGP signature

Bug#1059267: ITP: apt-verify - extend apt's gpgv-based verification mechanism

[Simon Josefsson]

Package: wnpp
Severity: wishlist
Owner: si...@josefsson.org
X-Debbugs-CC: debian-devel@lists.debian.org

* Package name: apt-verify
  Version : 2.0
  Upstream Contact: Simon Josefsson 
* URL : https://gitlab.com/debdistutils/apt-verify
* License : AGPLv3+
  Programming Lang: Shell script
  Description : extend apt's gpgv-based verification mechanism

Apt-verify extends apt to call all tools in /etc/verify.d/ instead of
always only calling gpgv, to verify apt archive integrity and
authenticity.  A symbolic link in /etc/verify.d/gpgv is installed by
default to provide full backwards compatibility.

/Simon


signature.asc
Description: PGP signature

Re: Signature strength of .dsc

[Simon Josefsson]

Jonathan McDowell  writes:

> On Mon, Dec 04, 2023 at 11:07:38AM +0100, Simon Josefsson wrote:
>> Judit Foglszinger  writes:
>> >> > Dmitri, could you re-run the numbers with the debian-maintainer
>> >> > keyring?
>> >> 
>> >> That is correct. I have updated the results now.  The 2,455 no
>> >> public key has now become 1,238
>> >
>> > Another is the DN keyring.  Also I'd expect many keys to be found in
>> > older versions of the keyring package/keyring repository and on
>> > keyservers like keyserver.ubuntu.com
>> 
>> Removing old keys is usually a bad idea -- could these be moved to a
>> "archived" keyring instead?  I assume having them in the "live"
>> keyring is not possible if the presence of a key in that file is used
>> to make authorization decisions.
>> 
>> You want to be able to verify old signatures in 20+ years too, and
>> then you need to be able to find the corresponding public key.
>
> For a long time we had a "removed" keyring, but we decided that we
> didn't want to continue shipping a keyring that was explicitly a set of
> keys we could not vouch for the trust of (whether that be because they
> were revoked, lost, weak, or whatever). If you really want to find old
> keys there is 15+ years of history in the keyring git repository, as
> Judit mentioned:
>
> https://salsa.debian.org/debian-keyring/keyring/

I think that is unfortunate and not sustainable over time: you need to
have access to the public keys to verify old signatures, and for as long
as the old signatures are published we should make a public keyring for
them easily available.  Which I suspect means essentially forever, due
to archive.debian.org.

I don't think it doesn't really matter of the old public key is weak or
invalid: if we know of a public key published at the time as some
signature that was possible to verify using software available at that
time, we should publish that public key.

Was there a real practical situations that couldn't be resolved that
lead to dropping the "removed" keyring?  What was the details?  Maybe
this decision could be reverted with some effort.

/Simon


signature.asc
Description: PGP signature

Re: Signature strength of .dsc

[Simon Josefsson]

Judit Foglszinger  writes:

> Hi,
>
>> > Dmitri, could you re-run the numbers with the debian-maintainer keyring?
>> 
>> That is correct. I have updated the results now.
>> The 2,455 no public key has now become 1,238
>
> Another is the DN keyring.
> Also I'd expect many keys to be found in older versions of the keyring 
> package/keyring repository
> and on keyservers like keyserver.ubuntu.com

Removing old keys is usually a bad idea -- could these be moved to a
"archived" keyring instead?  I assume having them in the "live" keyring
is not possible if the presence of a key in that file is used to make
authorization decisions.

You want to be able to verify old signatures in 20+ years too, and then
you need to be able to find the corresponding public key.

Even finding a copy of my own old RSA1280 key 0xB565716F turned out to
be tricky, I had to search for it just a couple of days ago and I
couldn't find it on the keyservers I looked on.  The key was used during
2002-2014 to sign a lot of software releases (and emails).  Fortunately,
I had a habbit of sticking it into AUTHORS field of some packages so I
found it here:

https://git.savannah.gnu.org/cgit/libidn.git/tree/AUTHORS?id=cd51d7cd4e83f8b5240517b63ba2adef721542c9

/Simon


signature.asc
Description: PGP signature

Re: Signature strength of .dsc

[Simon Josefsson]

Salvo Tomaselli  writes:

>> hi, on "no public key" list there are my uploads, I'm debian maintainer 
>> (https://nm.debian.org/person/fantu/), I signed with my key and I have 
>> DM upload right for them 
>> (https://qa.debian.org/developer.php?login=fantonifabio%40tiscali.it)
>
> I think he just didn't check the debian-maintainer keyring at all.

Dmitri, could you re-run the numbers with the debian-maintainer keyring?

The numbers suggest to me that signing strength of DSC signatures on the
contrary really do provide value and that it is working well.  The
instances of RIPEMD160/SHA1 I checked were old, and the numbers of
failures are quite low compared to overall number of uploaded packages.
Thus we have good assurance on the majority of packages.

We should make sure RIPEMD160/SHA1 signatures are rejected going
forward, as well as the wrong-key-usage.

/Simon


signature.asc
Description: PGP signature

Bug#1042827: ITP: priv-wrapper -- library to disable resource limits and other privilege dropping

[Simon Josefsson]

Package: wnpp
Severity: wishlist

Hi!  I am planning to package priv-wrapper:

   https://cwrap.org/priv_wrapper.html

   priv_wrapper aims to help running processes which are dropping
   privileges or are restricting resources in test environments. A
   disabled call always succeeds (i.e. returns 0) and does nothing. The
   system call pledge exists only on OpenBSD.

It is similar in spirit as the rest of the cwrap projects -- see
https://cwrap.org/ -- and several seems to be packaged in Debian:

https://tracker.debian.org/pkg/nss-wrapper
https://tracker.debian.org/pkg/uid-wrapper
https://tracker.debian.org/pkg/socket-wrapper
https://tracker.debian.org/pkg/pam-wrapper

The license appears to be GPLv3+.

/Simon


signature.asc
Description: PGP signature

Re: [RFC] changes to rsyslog - default to RFC 5424 format

[Simon Josefsson]

Michael Biebl  writes:

> Hi,
>
> we are early in the bookworm release cycle, so I guess it's the
> perfect time to bring up this topic.

Sorry for hijacking the thread, but perhaps now is a good time to stop
using the legacy syslog time format and use the standardized RFC 5424
format?  It is the default format in upstream rsyslog, but the default
Debian config uses the legacy format.

Effectively, the change that I suggest is to stop putting this into
/etc/rsyslog.conf by default:

#
# Use traditional timestamp format.
# To enable high precision timestamps, comment out the following line.
#
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

The legacy time format that is used today does not record year, timezone
or subsecond information.  Compare /var/log/syslog outputs like this:

Nov 23 21:47:31 latte jas: test

with

2021-11-23T21:47:49.082799+01:00 latte jas: test

/Simon


signature.asc
Description: PGP signature

Re: Debian choice of upstream tarballs for packaging

[Simon Josefsson]

Paul Wise  writes:

> Hi all,
>
> I noticed that sometimes Debian's choice of upstream source for
> packaging can be suboptimal. This is especially apparent for the
> different per-language upstream packaging ecosystems[1], where the
> upstream packaging differs from the upstream VCS in some significant
> ways, including missing files, prebuilt files, embedded copies etc.
>
> While the upstream VCS also sometimes has these issues, it is often
> much less problematic than the upstream packaging ecosystems.

While I agree with the points you raise, and think I agree with your
overall goal, I see some problems with using upstream VCS as a source
for Debian packaging:

1) Trust paths.  Some upstreams sign release tarballs with an OpenPGP
release key that Debian trust for making releases.  Not all upstream
uses the same key to sign VCS tags/commits, and not all upstreams sign
VCS tags/commits at all.  While Debian can encourage and promote new
policies for upstream here, I don't think we are in a position to
require any uniform set of rules.  Signing tarballs is the current
established best practice -- moving to VCS builds needs a set of new
schemes to be established and deployed, and I don't see any single
universal solution today.

2) Bootstrapping projects from VCS is complex and requires additional
tools, and I think the Debian packaging process is well suited for this.
Two examples that I have run into:

  2a) Gnulib.  Several GNU-related projects import files from gnulib
  during VCS bootstrapping, and the way this happens is different for
  different projects.  The correct version of the files must be imported
  in the right way for things to work, and knowledge of which gnulib
  version is used is not always present in VCS but only in a released
  tarball.  How would this work when packaged in Debian?  A debian
  package containing the gnulib git repository could be added, to allow
  source packages to checkout the right version during build.

  2b) Cross-compilation and dependency cycles.  Bootstrapping from VCS
  may require a lot of tools that are optional when building from
  tarball, and in my experience the complete set of tools to bootstrap a
  project is rarely added as Build-Dep to Debian packages.  I feel some
  additional package build dependency mechanism would help here: maybe a
  Build-Bootstrap-Dep header to list the tools needed to generate a
  Debian source package?  And Build-Dep could list the tools needed to
  build Debian binary packages from the Debian source package.  I admit
  my understanding of the Debian packaging system is quite limited
  though.

3) Bootstrappable builds.  I think the underlying goal when it comes to
building from VCS may be to achieve bootstrappable builds -- see
https://www.bootstrappable.org/ -- however it seems to me that a lot of
care has to be taken when moving from tarball builds to VCS builds so we
don't make it harder to re-bootstrap the entire toolchain.  For example,
building GNU Coreutils from a tarball works fine in extremely old
environments, but building GNU Coreutils from VCS requires modern tools,
and perhaps some of them doesn't support older environments any more.

/Simon


signature.asc
Description: PGP signature

Re: Planning for libidn shared library version transition

[Simon Josefsson]

Hi!  I'm now resuming work on the libidn shared library transition, and
I'm ready for the upload to experimental.  I wanted to ping back here to
get more review.  I'm following Andreas Metzler's outline, but included
some tweaks suggested by Simon McVittie.  I decided to do some more
changes that are unrelated here (e.g., drop the would-be poorly named
libidn11-java package which nobody appears to use, and drop the no
longer working Emacs libraries).

The old package (1.33-3, in bullseye) looks like the following (yeah,
the 'Replaces: libidn11-dev' on the 'libidn11' package is confusing, and
could potentially cause problems here):

Package: libidn11
Architecture: any
Depends: ${misc:Depends}, ${shlibs:Depends}
Conflicts: libidn9-dev
Replaces: libidn11-dev
Pre-Depends: ${misc:Pre-Depends}
Multi-Arch: same
Description: GNU Libidn library, implementation of IETF IDN specifications

Package: libidn11-dev
Section: libdevel
Architecture: any
Depends: libidn11 (= ${binary:Version}), pkg-config, ${misc:Depends}
Conflicts: libidn9-dev
Multi-Arch: same
Description: Development files for GNU Libidn, an IDN library

The new package (1.38-1) that I'm planning to upload has this:

Package: libidn12
Architecture: any
Depends: ${misc:Depends}, ${shlibs:Depends}
Pre-Depends: ${misc:Pre-Depends}
Multi-Arch: same
Description: GNU Libidn library, implementation of IETF IDN specifications

Package: libidn-dev
Section: libdevel
Architecture: any
Depends: libidn12 (= ${binary:Version}), pkg-config, ${misc:Depends}
Breaks: libidn11-dev (<< 1.38-1~)
Replaces: libidn11-dev (<< 1.38-1~)
Suggests: idn
Multi-Arch: same
Description: Development files for GNU Libidn, an IDN library

Package: libidn11-dev
Section: oldlibs
Architecture: any
Depends: libidn-dev (>= 1.38-1~), ${misc:Depends}
Multi-Arch: same
Description: Transitional development package for GNU Libidn

Libidn11-dev's '>= 1.38-1~' dependency, and the '<< 1.38-1~' in
libidn12, came as a suggestion from Simon McVittie.

It builds fine, and reverse dependencies builds too (the two failures
are not libidn-issues):
https://salsa.debian.org/debian/libidn/-/pipelines/270968

I'm including more complete debdiff below.

What do you think?

I'm not only looking to catch fatal mistakes, but also want ideas on how
to improve this further, so all feedback is welcome.

Thanks,
/Simon

jas@latte:~/dpkg$ debdiff --show-moved libidn_1.33-3_amd64.changes 
libidn_1.38-1_amd64.changes 
Warning: these package names were in the second list but not in the first:
--
libidn-dev
libidn12
libidn12-dbgsym


Warning: these package names were in the first list but not in the second:
--
libidn11
libidn11-dbgsym
libidn11-java

[The following lists of changes regard files as different if they have
different names, permissions or owners.]

Files only in first set of .debs, found in package idn
--
-rw-r--r--  root/root   /usr/share/doc/idn/AUTHORS.gz
-rw-r--r--  root/root   /usr/share/doc/idn/TODO
-rw-r--r--  root/root   /usr/share/emacs/site-lisp/idna.el
-rw-r--r--  root/root   /usr/share/emacs/site-lisp/punycode.el

Files only in first set of .debs, found in package idn-dbgsym
-
-rw-r--r--  root/root   
/usr/lib/debug/.build-id/b8/51518b886fad3ef49ddda44ba667ac043e60c0.debug

Files only in first set of .debs, found in package libidn11
---
-rw-r--r--  root/root   /lib/x86_64-linux-gnu/libidn.so.11.6.16
-rw-r--r--  root/root   /usr/share/doc/libidn11/changelog.Debian.gz
-rw-r--r--  root/root   /usr/share/doc/libidn11/changelog.gz
-rw-r--r--  root/root   /usr/share/doc/libidn11/copyright
lrwxrwxrwx  root/root   /lib/x86_64-linux-gnu/libidn.so.11 -> libidn.so.11.6.16

Files moved from package libidn11 to package libidn12
-
-rw-r--r--  root/root   DEBIAN/shlibs
-rw-r--r--  root/root   DEBIAN/symbols
-rw-r--r--  root/root   DEBIAN/triggers

Files only in first set of .debs, found in package libidn11-dbgsym
--
-rw-r--r--  root/root   
/usr/lib/debug/.build-id/14/d4bc0771a40445c675c5c4682d56465463ccc8.debug
lrwxrwxrwx  root/root   /usr/share/doc/libidn11-dbgsym -> libidn11

Files only in first set of .debs, found in package libidn11-dev
---
-rw-r--r--  root/root   /usr/share/doc-base/libidn11
-rw-r--r--  root/root   /usr/share/doc/libidn11-dev/examples/README
-rw-r--r--  root/root   /usr/share/doc/libidn11-dev/examples/example.c
-rw-r--r--  root/root   /usr/share/doc/libidn11-dev/examples/example2.c
-rw-r--r--  root/root   /usr/share/doc/libidn11-dev/examples/example3.c
-rw-r--r--  root/root   /usr/share/doc/libidn11-dev/examples/exampl

Re: Planning for libidn shared library version transition

[Simon Josefsson]

Simon McVittie  writes:

> On Wed, 26 May 2021 at 19:18:24 +0200, Simon Josefsson wrote:
>> Andreas Metzler  writes:
>> > Why not use a versioned Provides *instead* of the dummy package?
>> 
>> Yeah, I never understand exactly when these dummy packages are needed.
>
> My understanding is that they are usually still necessary for smooth
> upgrades from one stable release to the next, because apt's dependency
> resolver tries not to remove packages (to minimize risk of removing
> something you were relying on), and removing a "real" libidn11-dev
> package because its lockstep version dependency is no longer satisfiable
> will still count as removing a package, even if a new libidn-dev now
> Provides it.
>
> It's a probabilistic thing, because apt calculates a "score" for various
> options for the whole upgrade transaction and chooses the one with the
> highest score - on some systems it will be able to figure out the right
> upgrade transaction even without a transitional package, but on others it
> will not. Having a transitional package makes it more likely that the
> intended upgrade happens, because apt gives a high "score" to upgrading a
> package that was already installed.
>
> piuparts only routinely tests relatively small installations - there are
> periodic QA tests done on larger systems like the union of all desktop
> tasks, but those are more expensive to run.

Thanks -- you convinced me to prefer libidn is not a guinea pig for
testing something like this.  I do think it would be a wortwhile release
goal, at some point, to fix tooling so that dummy packages are no longer
necessary for package transitions.  The information to make proper
decisions for 'apt' should be there without using dummy packages, as far
as I understand.  On the other hand, there are probably more important
areas (say, 100% reproducible builds) to work on instead.

/Simon


signature.asc
Description: PGP signature

Re: Planning for libidn shared library version transition

[Simon Josefsson]

Andreas Metzler  writes:

> On 2021-05-26 Guillem Jover  wrote:
>> On Tue, 2021-05-25 at 19:43:21 +0200, Andreas Metzler wrote:
> [...]
>> I'd probably instead make this a versioned Provides, so that the
>> transitional package can be removed right away from systems, it does
>> not interfere with the transition, and people can switch to the new
>> package in parallel w/o disruption.
>
> Hello,
>
> Why not use a versioned Provides *instead* of the dummy package? We have
> had these a long time our packagment management system should have 100%
> support now.

Yeah, I never understand exactly when these dummy packages are needed.
Before I remembered about dummy packages, I tested a libidn update
without it, and it appeared to build reverse dependencies just fine
(piuparts/reprotest).  The only answer I have seen is that 'some of our
old tooling behaved strange' if you didn't have a dummy transitional
package, without specific references to actual tools or versions.

Maybe we can try a transition without a dummy package, and use it as an
experiment to see what breaks?  I suspect that after a release, it is a
good time to do it.

/Simon


signature.asc
Description: PGP signature

Re: Planning for libidn shared library version transition

[Simon Josefsson]

Andreas Metzler  writes:

> On 2021-05-24 Simon Josefsson  wrote:
>> Hi!  This is for post-bullseye, but I appreciate guidance if anyone has
>> time.  Shared library version transitions trigger uncertainty in me.
>
>> I want to upload a new upstream libidn release into Debian, but upstream
>> has done a shared library transition.
>
> Hello Simon,
>
> I will assume that "shared library transition" means that that libidn
> 1.34 will have a soname bump (libidn12.so) but that the new release is
> API compatible. The following assumes this is true ...

Hi.  Thanks for review!

For all practical matters, yes, it is completely API compatible, the
only thing that changed was this struct (that should never have been in
the public .h anyway) that changed from:

  struct Stringprep_table
  {
Stringprep_profile_steps operation;
Stringprep_profile_flags flags;
const Stringprep_table_element *table;
  };
  typedef struct Stringprep_table Stringprep_profile;

into:

  struct Stringprep_table
  {
Stringprep_profile_steps operation;
Stringprep_profile_flags flags;
const Stringprep_table_element *table;
size_t table_size;
  };
  typedef struct Stringprep_table Stringprep_profile;

Arguably, doing a soname bump for this minor change could probably have
been avoided, but this was back in 2018 and other distributions have
upgraded to newer libidn.  If we revert the soname upstream now, it will
cause a lot of problems for others, so I don't think that is a good
idea.

We could patch upstream in Debian to avoid the soname bump, but that is
rather confusing.  It is good timing now to fix this once bullseye is
released, so I think we should just do that.

>> Bullseye will ship with
>> libidn11-dev instead of libidn-dev too.  Is the first step on the
>> transition to provide a libidn-dev package experimental (and after the
>> release, unstable) and make all reverse build-dependencies use it?
>
> That would delay the whole thing unnecessarily, you would not want to
> artificially make changing all rdeps a pre-dependency of your work
>
> First off if the new version of libidn is API compatible with 1.33 there
> will be no hard /requirement/ for renaming libidn11-dev to libidn12-dev.
> You could have libidn11-dev as a dev-package for libidn12. It is not
> aesthetically pleasing and confusing but would continue to work and has
> the least potential for error. OTOH a soname bump is good time for
> renaming the development package since you will have a new binary
> package (libidn12) and will need to go through NEW processing anyway.
> However you should do soname bump and  dev-package rename in one upload
> (to experimental) instead of beforehand. This ways ftp-master will only
> need to check it once. So you would use this timelime:
>
> 1. Prepare packages of the new upstream targeted for experimental
> (libidn12, libidn-dev, libidn11-dev transition package). Test.
> 2. Upload to experimental.
> 3. Wait for NEW processing
> 4. Do some more tests, once you thinks all rdeps could be built against
> libidn12 you request a transition slot. Please note that at this point
> no source changes to rdeps related to packaging changes needed to happen,
> they can (and should) continue to b-d on libidn11-dev which pulls in the
> libidn-dev.
> 5. debian-release gives you the GO for uploading the package to testing.
> 6. You do this and debian-release triggers rebuilds of all rdeps.
> 7. All rebuilt rdeps and libidn12/libidn-dev/libidn11-dev propagate to
> testing together 10 days later.
> 8. Now you can start submitting bug-reports against rdeps asking for a
> change of the b-d.

That (and your suggestions below) seems good.  I like it better than my
approach.  I'll start to implement this and post a debdiff for review
before actually making the upload into experimental.

/Simon

> [...]
>> A 'git diff' between the version in bullseye and what I want to upload
>> to experimental is shown below, together with debdiff-output between the
>> built packages.
>
>> Generally, does things looks okay?  Specifically, what about the
>> Breaks/Replaces/Conflicts?  The d/changelog entry?  Will the confusing
>> 'Replaces: libidn11-dev' for the libidn11 (!) package in bullseye cause
>> any problem?  Am I right to drop it here?
>
>>  Package: libidn11-dev
>> +Section: oldlibs
>> +Architecture: any
>> +Depends: libidn-dev, ${misc:Depends}
>
> I would use (= ${binary:Version}) for the depends to make sure that
> libidn11-dev 1.38 together with libidn-dev 1.34 do not fullfill a
> dependency on libidn11-dev (>= 1.35)
>
>> +Package: libidn-dev
>>  Section: libdevel
>>  Architecture: any
>>  Depends: libidn11 (= ${binary:Version}